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Executive Summary 



Achieve is an independent, bipartisan, nonprofit organization created by governors and corporate 
CEOs to help raise academic standards and achievement in the nation’s public schools. As a 
service to states, Achieve provides independent analyses of the quality and coherence of their 
curriculum, standards, assessments and accountability systems. Since 1998, Achieve has 
provided such evaluations to 14 states, comparing their systems to the best models, or 
“benchmarks,” from the United States and abroad. 

At the request of Superintendent Jerry Weast, Achieve conducted a similar evaluation of the 
Montgomery County Public Schools’ (MCPS’) K-12 Curriculum Frameworks and high school 
semester exams in English and mathematics. The review was designed to determine how well the 
MCPS Frameworks and exams align with the state of Maryland’s academic standards, and, more 
broadly, how they compare with exemplary standards from other states and nations. 

The context for this work is an increasingly ambitious set of goals the MCPS leadership has set 
for all Montgomery County schools and students, spurred in part by higher standards the state 
has set and by the desire to close the achievement gap that has left some students ill prepared for 
the opportunities and challenges that await them. The county can no longer be satisfied if some, 
or even most, students are performing well. The goal is for every school to help every student 
achieve higher standards. 

If the MCPS Curriculum Frameworks and assessments are to serve as the foundation for the new 
accountability system, they must be of very high quality and they must be accessible and useful 
to parents, teachers, principals and others. Achieve’ s analysis was designed to answer a series of 
key questions about the quality and utility of MCPS Frameworks and exams. The overarching 
questions and findings are summarized below. 

• Do the MCPS Curriculum Frameworks in English/language arts and math adequately 
address the essential knowledge and skills of each discipline? 

Overall, Montgomery County’s Curriculum Frameworks in English and math are quite 
comprehensive. With a few minor exceptions, the core knowledge and skills that one would 
expect to find in high-quality curriculum guides are present in the district’s Frameworks. 

• Are the Curriculum Frameworks designed in a way that is easily understood and 
accessible to teachers and school leaders? Are the standards clearly articulated and 
measurable? 

For the most part, the expectations in the Frameworks are clearly articulated and jargon- free. 
Moreover, their format and numbering schema make them easy to use. They avoid the common 
shortcoming that other states and districts have struggled with over the last several years — 
overly broad expectations that do not provide enough guidance to teachers. Instead, Montgomery 
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County has chosen to be explicit about the knowledge and skills students should learn in each 
grade. That specificity contributes to clarity, which in turn increases the chances that teachers 
across the district are holding students to a common, high expectation. There are, however, 
several areas where clarity and precision can be improved, particularly in English/language arts. 

• How do the MCPS Curriculum Frameworks compare with Maryland’s standards? 

The county’s Frameworks in English/language arts and math are very consistent with Maryland’s 
Content Standards (K-8) and Core Learning Goals (9-12). However, while the state developed 
content standards only for key grades — 3,5,8 and 12 — Montgomery County specified grade- 
by-grade expectations from prekindergarten through 8 th grade, further developing, clarifying and 
sometimes adding to the Maryland Content Standards. The result is a set of county preK-8 
standards that are aligned to, but not limited by, the state standards. Achieving alignment 
between county and state documents at the high school level was a more challenging task for the 
district because Maryland’s High School Core Learning Goals are structured differently from its 
K-8 Content Standards and function more as blueprints for the high school tests than as courses 
of study. 

• How do the MCPS Frameworks compare with those of exemplary states and nations? Are 
expectations for schools and students rigorous yet reasonable? 

The MCPS Frameworks are rigorous and reasonable. If they are strengthened in several key 
areas, they can be on par with the best in the nation and the world. The MCPS “System of Shared 
Accountability” expects that, at a minimum, no graduate will enter college required to take 
remedial courses and that most students will actually complete college-level courses prior to high 
school graduation. In order to realize this ambitious goal, Achieve recommends that the county 
continue to raise the rigor of its Frameworks by improving specificity and progression in 
English/language arts and by clarifying and refining the content of its algebra and number 
standards in mathematics, particularly in middle and high school. 

• How well do the MCPS high school semester tests in English 9, Algebra I and Geometry 
measure the knowledge and skills laid out in Maryland’s Core Learning Goals? Are they 
predictive of student success on the end-of -course tests that will soon be a statewide 
graduation requirement? 

Montgomery County’s semester tests are high quality and align to both the district Frameworks 
and the state standards. As a result, students who perform well on these tests are likely to 
perform well on the corresponding state tests. 

• How challenging are the MCPS high school tests? Are they sufficiently rigorous that 
students passing these tests will be prepared to complete more advanced courses? 

The MCPS tests generally are rigorous and assess important high school content. However, if the 
goal is to have most students prepared for college-level work by 12 th grade — such as AP 
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English and Calculus — the district should raise the level of rigor of its English 9 “honors” level 
first semester test and its Algebra I and Geometry tests. 

* * 

It is evident that Montgomery County has made remarkable progress in creating, expanding and 
continually improving an aligned system of Curriculum Frameworks, instructional guides and 
diagnostic tests. There is more work to do, to be sure, but this careful attention to high standards 
and capacity building should go a long way toward helping teachers and students understand and 
meet the intellectual demands of the 21 st century. 
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Introduction: Raising Standards in America’s Schools 



Since the release of A Nation at Risk in 1983, schools, states and national policymakers have focused 
increasingly on improving the level of academic achievement of all students. At that time, most 
school systems awarded diplomas based on Carnegie Units, which generally represented “seat time” 
as opposed to a demonstration of essential knowledge and skills. Under the Carnegie, or credit-based, 
system, postsecondary institutions and employers had difficulty determining what students had 
learned, as course content varied from school to school. Additionally, it became increasingly clear 
that a large percentage of our nation’s students, particularly minorities and the poor, were being 
dramatically underserved by the education system because they were not provided with the same rich 
curriculum and learning opportunities as their more affluent counterparts. 

In an attempt to raise the level of student learning across the board and create a more publicly 
accountable education system, states, school districts and national organizations began the process of 
articulating academic standards to define what all students should know and be able to do. In 
Montgomery County and across the country, standards now are the driving force in efforts to 
improve equity and excellence. In 1996, only 14 states had standards. Today, 49 states have both 
standards and assessment systems to measure achievement of those standards. Most also plan to hold 
their schools and students accountable for performance on the assessments. By stating clearly the 
knowledge and skills students are expected to gain as a result of their schooling, state and local 
leaders hope that students will better understand what is expected of them, schools will improve their 
programs to help students achieve those expectations, and low-performing districts and schools in 
particular will be challenged to raise the level of teaching and learning. 

Because states have made substantial investments in the new standards and tests — and because 
accountability systems make these standards and tests matter — policymakers and the public want to 
know how their standards compare to what other states and countries expect. They also want to know 
if the testing and accountability systems are as strong as those of successful states and whether they 
will accomplish their goals of helping all schools and students reach high standards. Achieve was 
created precisely to address these issues. 

Born out of the 1996 National Education Summit, Achieve helps states ensure that they have 
established: 

• Standards that compare favorably with the academic expectations of other states and high- 
performing nations; 

• Assessments that accurately measure student achievement against those standards; and 

• Policies that provide appropriate information, incentives and support for schools and the 
public. 

Achieve is an independent, bipartisan, nonprofit organization overseen by a board of directors 
composed of governors and corporate CEOs. Achieve serves as a clearinghouse and resource center 
on standards, testing and accountability. Achieve works primarily with states to support their work to 
improve their schools. 
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Benchmarking to the Best 



To help states and school districts in their efforts to improve schooling for all students, Achieve 
provides a benchmarking sendee. Through benchmarking. Achieve compares a state’s or district’s 
standards, assessments and accountability system to the best available models from the United States 
and abroad. Benchmarking is a highly respected practice in the business world. It is an activity that 
looks outward to find best practices and high performance, and then measures actual business 
operations against those examples. Benchmarking in education follows the same principle. At a time 
when education reforms are focused on raising standards and performance, it is appropriate that 
states and districts seek an external yardstick to gauge their efforts. 

Through more than five years of work with states, Achieve has developed a firm idea of what strong 
academic standards look l ik e. They are clear and specific enough to guide curriculum planning. They 
set rigorous, yet reasonable, expectations for all students and raise the bar higher than it currently is 
set for many students. They integrate content knowledge with important thinking skills. And they are 
widely read and understood by parents, educators, businesspeople and policymakers. 

Achieve also has a clear sense of what strong assessments look like. They are aligned tightly with 
standards and, therefore, accurately measure the curriculum. They are rigorous and incorporate a mix 
of question formats, including multiple choice, short answer and extended response items such as 
essays. And they provide concrete information to educators, parents and the general public that can 
be used to raise the achievement of every child. 

By benchmarking standards, assessments and accountability policies, Achieve hopes to help states 
and districts answer the following questions: 

• How do our standards compare with those of other high-performing states and nations? 

• Are the expectations for our students and schools high enough? 

• How well do our assessments measure the knowledge and skills laid out in the standards? 

• Do our accountability policies provide appropriate incentives for students, teachers and 
administrators? 

Achieve is involved in benchmarking for another important reason: States and districts traditionally 
have had limited access to high-quality, comparative information about education policies and 
practices. This is due partly to the fact that the standards movement in education is relatively young. 
But it also is a result of the disparate nature of much of the work that has been done to date. While 
the standards reviews and “report cards” issued by other organizations have helped to focus national 
attention on the quality of standards, their judgments often have been in conflict. States and districts 
increasingly are looking for independent, credible and constructive advice on these issues. 
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Working with Montgomery County Public Schools 



Superintendent Jerry Weast has set out an ambitious agenda for the Montgomery County Public 
Schools. He has challenged every school to help every student achieve higher standards. Schools 
can no longer be satisfied if some, or even most, students are performing well. 

The MCPS accountability system is based on 100 percent of the county’s students being 
prepared for college-level work by graduation, if not prior to it. The goal is to raise the floor for 
all students while raising the ceiling for those who traditionally have achieved at higher levels. 
The county curriculum and tests are intended to help achieve this goal. 

As MCPS gears up to meet this challenge, it must do so in the context of evolving federal and 
state education policies. The federal No Child Left Behind Act will require Maryland, and every 
other state, to revamp its testing system and to establish more stringent accountability measures. 
By 2006, the state must test students in reading and mathematics in every grade from 3 to 8 and 
at least once in high school. By 2007, it must test students in key grades in science. Test results 
will be disaggregated by race, socioeconomic status and other characteristics, and schools will be 
held accountable for raising achievement for all groups of students every year. 

In responding to the new federal law, Maryland is rebuilding its testing system. The Maryland 
State Performance Assessment Program (MSPAP) is being replaced by a new series of tests 
designed to align with the state’s academic standards and to provide results for individual 
students (something MSPAP did not do). Also, the state has put in place a more challenging set 
of end-of-course exams that students in the class of 2007 will have to pass in order to graduate. 

If all Montgomery County schools are going to rise to the challenge of the new state and district 
goals, the county’s curriculum must be of very high quality and its assessments must be accurate 
measures of that curriculum. Given Achieve’ s experience reviewing standards and assessments 
in 14 states, Superintendent Weast asked Achieve to conduct an independent evaluation of the 
MCPS Curriculum Frameworks and high school semester exams with two goals in mind: a 
comparison of the county’s Frameworks with Maryland standards and exemplary state and 
international standards; and an analysis of the county’s semester tests to determine how well they 
assess Maryland’s standards. A summary of Achieve’ s findings, as well as more detailed 
treatment of observations and recommendations, follows. 

To analyze the MCPS Curriculum Frameworks, Achieve senior staff and national experts in 
English/language arts and math compared the content and skills found in the county’s 
Frameworks to Maryland’s Content Standards and Core Learning Goals, and to Achieve’ s 
“benchmark” standards from other states and nations (California and Massachusetts in 
English/language arts; New Standards, North Carolina and Texas in early literacy; and Arizona, 
Japan and Achieve’s end-of-grade 8 expectations, Foundations for Success, in mathematics). 
They looked at how clearly the standards are articulated, how rigorous they are, and how 
effective they are at establishing a progression of knowledge and skills through the grades that 
will prepare all MCPS students to be “college ready.” Analyzing the high school semester exams 
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was a separate, but complementary, process that involved the systematic application of a 
protocol — a multistep process based on defined criteria for evaluating alignment — specifically 
designed to reveal how well a test measures a set of related standards (in this case, Maryland’s 
Core Learning Goals). Achieve convened two teams of highly skilled subject-matter educators to 
study the tests and apply the Achieve protocol. The reviewers are a deliberate mix of classroom 
teachers, and curriculum and subject matter specialists, each with extensive expertise in content 
and assessment design. They have experience in large-scale assessments and/or standards 
development; represent a diversity of viewpoints on curriculum matters; and have worked in a 
variety of challenging school environments in rural, suburban and urban settings. This diversity 
of backgrounds has proven invaluable in arriving at considered judgments (for example, 
determining the appropriateness of a test item for a particular grade level). Achieve’ s senior 
consultants in English/language arts and math led their teams through a stepwise application of 
Achieve’ s protocol with the goal of arriving at consensus judgments for each of the alignment 
criteria. Such interaction prevents snap judgments and fosters thorough evaluation. 

In writing this report, Achieve synthesized the expert reviews of the county’s Curriculum 
Frameworks and summarized the results of the alignment studies as reported by the review teams 
in English/language arts and mathematics. The findings described in this report represent 
consensus opinions of Achieve’s consultants and experts, but final judgments and conclusions 
rest with Achieve. In addition to this summary report, Achieve prepared a detailed technical 
report for the Superintendent and his instructional staff. Because the technical report contains 
references to secure test items, it is confidential. 

Brief biographies of experts and consultants who participated in Achieve’s standards 
benchmarking and assessment analysis for MCPS can be found in the Appendix. 
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The Achieve Benchmarking Methodology 



Achieve’ s benchmarking and alignment methodology has been developed, tested and refined 
over four years. Achieve staff and consultants have reviewed dozens of sets of standards and 
tests and bring their experience to bear in applying these tools to MCPS standards and tests. 

Standards Benchmarking 

Achieve compares a state’s or district’s standards to state, national and international benchmark 
standards recognized for their quality and/or for producing high student achievement. This 
comparison of state or district standards to benchmark standards is designed to answer the 
following questions: 

• Do the standards define a comprehensive, yet manageable academic core for all students? 
Are there key concepts or skills missing in the standards? 

• Do the standards convey both the breadth and depth of knowledge and skills expected of 
students? Have choices been made about what is most important for students to learn and 
when? 

• Are the standards as rigorous as they should be? Do the expectations described by the 
state’s or the district’s standards compare favorably to those of the benchmark standards? 

• Do the standards define both what students should know and what they should be able to 
do with that knowledge? For example, do the standards explicitly describe both 
mathematical concepts and the level of reasoning and problem-solving skills students are 
expected to apply? Or is one overemphasized at the expense of the other? 

• Are the standards clear and explicit, conveyed in a way that educators and parents can 
understand and use to improve student achievement? 

To ensure that the benchmark standards documents used as exemplars are indeed the best for this 
purpose, in early 1999, Achieve commissioned expert reviews of a variety of sets of standards. 
Reviewers concluded that the English/language arts standards from California (1997) and 
Massachusetts (2001), and the mathematics standards from Arizona (1996) and Japan (1990) had 
the most value for benchmarking. 1 Subsequently, Achieve augmented its benchmark standards, 
adopting the early literacy standards of North Carolina (1999), Texas (2001) and New Standards 
(1991), and Foundations for Success (2002), Achieve’s document detailing the mathematics that 
we believe all students should be expected to know before leaving 8 th grade. Developed by a 
panel of leading research mathematicians and math educators, and based on a close analysis of 
the curricular expectations in the highest performing European and Asian nations, Foundations 



1 Achieve created “benchmark profiles” for each of these documents that provide contextual information about the 
standards and summarize their strengths and weaknesses. The profiles are available upon request. 
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for Success tells us where we need to aim if we want the next generation of America’ s young 
people to be internationally competitive in their mathematical knowledge and understanding. 

Selecting these benchmarks proved to be a difficult task because no one set of standards is 
perfect, and judgments about the quality of standards are in some ways subjective. Still, we are 
confident that the choices used in our current work reflect some of the best thinking from around 
the country, and that a careful comparison of a state’s or district’s standards to these benchmarks 
will yield helpful diagnostic information and policy suggestions for states and districts to 
consider. 

For example, in the area of English/language arts, California distinguishes those standards 
concerned with informational text from those concerned with literature. This makes it easy for 
teachers to help students grasp the different strategies used in reading, understanding and writing 
the two types of text. Massachusetts organizes its standards in grade spans of two years, as do a 
number of states, but Massachusetts articulates its expectations for student learning more 
precisely than do most states. In mathematics, Arizona’s standards provide a level of specificity 
and detail that is helpful for teachers — especially for elementary generalists — while Japan’s 
standards provide an economical, yet focused and rigorous treatment of the discipline. Achieve’ s 
end-of-grade 8 expectations, described in Foundations for Success, feature sample problems and 
methods for solving the problems to help teachers understand what the expectations really mean. 
The problems underscore the depth of mathematical understanding and reasoning skills implied 
by the expectations. 

Assessment-to-Standards Alignment Analysis 

Achieve’ s assessment-to-standards analysis is designed to address the alignment of tests to 
standards. It helps uncover answers to the following issues: 

• Fairness. Does each assessment only measure content and skills reflected in the 
standards? Or, put differently, can everything on the test be found in the standards? 

• Balance. Does each assessment measure the breadth and depth of content and skill in the 
standards? In other words, to what extent does each assessment measure the key content 
and skills for a grade level or course? 

• Rigor. Overall, is each assessment sufficiently challenging for students? Do the 
assessments become more complex from grade to grade? 

Alignment is not a “yes or no” question — nor is it a mathematical calculation. It is the extent to 
which standards and assessments are in agreement and serve in conjunction with one another to 
guide and support student learning. Consequently, responding to the above questions requires a 
systematic procedure to probe the different factors that contribute to alignment. Achieve’ s 
process, or “protocol,” for determining the alignment of assessments to standards is based upon 
four criteria. Application of the protocol provides rich information about alignment of tests and 
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standards, which typically is unavailable to districts and states. The fundamental criteria for 
Achieve’ s alignment process are: 

• Confirmation or construction of test blueprint. Reviewers check to see that each item 
corresponds to at least one standard or objective. If the state, district or test company fails 
to provide a test blueprint, then Achieve’s reviewers construct one. If the blueprint 
supplied by the test developer does not stand up to scrutiny (i.e., a significant number of 
items mapped to one standard or objective are found to be more closely related to a 
different one), reviewers may decide to revise the blueprint. 

• Content centrality. This criterion examines the quality of the match between the content 
of each test item and the content of the related standard. Reviewers determine how 
closely the content of the item matches that of the related standard, and then assign the 
item to one of four categories based on degree of alignment, from “not aligned” to 
“clearly aligned.” 

• Performance centrality. Each item places a certain type of cognitive demand on a 
student (e.g., the student is asked to “identify” or “analyze”). If an item simply requires a 
student to “identify” and the corresponding standard requires a student to “analyze,” then 
there is a mismatch between the two performances. Reviewers assign each item to one of 
four categories based on the degree of alignment, from “not aligned” to “clearly aligned.” 

• Challenge. This criterion is applied both to the individual items and the set of items that 
measure an entire strand, such as “Measurement.” Its purpose is to determine whether 
doing well on these items requires students to master challenging subject matter. At the 
item level, reviewers consider two factors related to challenge: source of challenge and 
level of cognitive demand. At the item-set level, reviewers consider the overall level of 
challenge of the items mapped to a strand. 

o Source of challenge. This criterion attempts to uncover whether an individual test 
item is “fair.” Reviewers analyze whether an item is difficult because of the 
knowledge and skills it targets — or for other reasons not related to the subject 
matter, such as relying unfairly on students’ background knowledge — and rate each 
item as having an appropriate or inappropriate source of challenge. Any item judged 
to have an inappropriate source of challenge is not examined when item sets are 
evaluated for level of challenge. 

o Level of cognitive demand. This criterion focuses on the type and level of thinking 
and reasoning required by the student on a particular item. 2 Level 1 items in 
English/language arts test literal recall or basic comprehension. In math they require 
students to recall information such as a fact, definition, term or simple procedure. 
Level 2 items call for the engagement of some mental processing beyond a habitual 



2 Norman L. Webb. (2001) Levels for Determining Depth of Knowledge. CCSSO TILSA Alignment Study, Version 
2.0, May 21-24, 2001. 
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response. In English/language arts, Level 2 items ask students to make inferences, 
while in math they require students to make some decisions as to how to approach a 
task or problem. Level 3 items in English/language arts require students to interpret 
text, while in math they require students to reason, plan or use evidence. Level 4 
items are constructed-response items, typically requiring extended time. In 
English/language arts these items require analysis and in math they demand complex 
reasoning, planning, developing and thinking. (Level 4 items generally are not 
included on large-scale, on-demand tests.) 

o Level of challenge. This criterion applies to the set of items that maps to a given 
standard. Reviewers compare the overall demand encompassed by a set of items to 
the level of demand expressed in the standard itself. In addition to evaluating 
alignment, reviewers also judge whether the set of test items has a span of difficulty 
appropriate for students at a given grade level. This judgment is based on the 
standards, the assessment and supporting materials such as student responses. 
Reviewers write a succinct summary of the level of challenge of each item set. 

o Balance and range. This criterion also is applied to the set of items that map to a 
standard. It compares the extent to which the knowledge and skills delineated in the 
standard receive the same emphasis on the assessment, and determines if that 
emphasis is appropriate. Range is a quantitative check on the degree to which the 
items sample the knowledge and skills described in the standards, since it is very 
difficult for one assessment to measure the full complement of knowledge and skills 
required by state standards. Evaluating balance and range provides both qualitative 
and quantitative information about the choices states or test developers have made. 

The results of Achieve’ s review of MCPS academic standards and assessments are organized 
into three sections. The first section summarizes the overall findings, while the second and third 
sections provide more detailed results for English/language arts and mathematics, respectively. 
As noted earlier, Achieve also will provide the district with a detailed technical report containing 
more fine-grained data from the analysis. Because that report references secure test items, it is 
presented confidentially. 
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Summary of Findings 



The Achieve review was designed to address an overarching set of questions about the 
academic expectations the county has set for its students and schools: Do the Curriculum 
Frameworks clearly spell out the essential knowledge and skills in each subject area and 
provide a basis for a robust, challenging instructional program that builds from grade to 
grade? How do the Frameworks compare with the Maryland Content Standards and the best 
from other states and nations? Are the county’s semester tests of high quality and well 
aligned to the state’s end-of-course tests? 

This section summarizes answers to the questions Achieve’ s review addressed. Each of the 
issues dealt with here is examined more thoroughly in the English/language arts and math 
findings in the two sections following the summary. 

• Do the MCPS Curriculum Frameworks in English/language arts and mathematics 
adequately address the essential knowledge and skills of each discipline? 

Overall, Montgomery County’s Curriculum Frameworks in English and math are 
comprehensive. The core knowledge and skills that one would expect to find in high- quality 
curriculum guides are present in the district’s Frameworks. 

Essential concepts and skills in reading, writing, English language conventions, listening and 
speaking all are addressed in the English/language arts preK-8 Framework. This finding 
holds true for mathematics as well; the principle domains are addressed and reviewers 
flagged only a minimal number of missing content and skill expectations. 

• Are the Curriculum Frameworks designed in a way that is easily understood and 
accessible to teachers and school leaders? Are the standards clearly articulated and 
measurable? 

For the most part, the expectations in the Frameworks are articulated clearly and are jargon- 
free. Moreover, their format and numbering scheme make them easy to use. They avoid the 
common shortcoming that other states and districts have struggled with over the last several 
years — overly broad expectations that do not provide enough guidance to teachers. Instead, 
Montgomery County has chosen to be explicit about the knowledge and skills students 
should learn in each grade. That specificity contributes to clarity, which in turn increases the 
chances that teachers across the district are holding students to a common, high expectation. 

The MCPS indicators have an exceptional feature in that they include examples to clarify the 
expectations. Achieve has reviewed many sets of state standards in which this essential 
feature is lacking. These concise illustrations make the district standards much more concrete 
and go a long way in demystifying the performances expected of students. We would 
encourage MCPS to amplify this feature by anchoring English/language arts examples with 
specific references to recommended texts identified in the district’s instructional guides and 
by expanding the repertoire of math problems. MCPS also might consider disseminating 
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examples of a variety of test items with related scored samples of student work to leave no 
doubt as to the quality of work expected. 



There are, however, several areas where clarity and precision can be improved. Although 
most preK-8 English/language arts indicators are not difficult to understand, there are some 
problems with consistency in the language used in indicators across grades. Some indicators 
are difficult to interpret with certainty. In addition, while most of the English/language arts 
indicators are framed in measurable terms (i.e., students are asked to identify, analyze, 
evaluate and distinguish), a few are focused more on process than results, making them 
difficult to assess. Indicators that include such words as “examine,” “explore” and 
“investigate” should be recast so that they define what students should know and be able to 
do once they have examined, explored or investigated something. 

The MCPS math indicators usually are clear, succinct and measurable and typically are 
broken out more precisely than those in English/language arts. 

• How do the MCPS Curriculum Frameworks compare with Maryland’s standards? 

The county’s Frameworks in English/language arts and math are very consistent with 
Maryland’s Content Standards (K-8) and Core Learning Goals (9-12). However, while the 
state developed Content Standards only for key grades — 3, 5, 8 and 12 — Montgomery 
County specified grade-by-grade expectations from prekindergarten through 8 th grade, further 
developing, clarifying and sometimes adding to the Maryland Content Standards. The result 
is a set of county preK-8 standards that are aligned to, but not limited by, the state standards. 

Achieving alignment between county and state documents at the high school level was a 
more challenging task for the district because Maryland’s High School Core Learning Goals 
are structured differently from its K-8 Content Standards and function more as blueprints for 
the high school tests than as courses of study. Despite the “disconnect” between the state’s 
K-8 standards and high school goals, the district did a credible job of capturing the 
significant aspects of the goals in the Curriculum Framework for its high school courses in 
English 9, Algebra I and Geometry. 

• How do the MCPS Frameworks compare with those of exemplary states and nations? 
Are expectations for schools and students rigorous yet reasonable? 

The MCPS Frameworks are rigorous and reasonable. If they are strengthened in several key 
areas, they can be on a par with the best in the nation and the world. The MCPS “System of 
Shared Accountability” expects that, at a minimum, no graduate will enter college required 
to take remedial courses, and most students will actually complete college-level courses prior 
to high school graduation. To help ensure that Montgomery County realizes its ambitious 
goal, Achieve recommends that the county continue to raise the rigor of its Frameworks by 
improving specificity and progression in English/language arts and by clarifying and refining 
the content of its algebra and number standards in mathematics, particularly at the middle 
and high school levels. 
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Rigor is a multifaceted condition. It is a function of clearly specifying essential content, 
ensuring the essential content is targeted at the right grade levels, and sequencing significant 
knowledge and skills so the level of complexity builds in a thoughtful and deliberate way 
from prekindergarten through 12 th grade. 

Although MCPS indicators in English/language arts generally articulate an appropriate 
trajectory of essential knowledge and skills from grade to grade when compared to Achieve’ s 
benchmark standards, some areas warrant further attention. For example, essential skills such 
as phonemic awareness and vocabulary study could be made more explicit. In other 
instances, district indicators use slightly different wording from grade to grade, but these 
differences do not make increased demands as clear as they could be. 

By high school, one expects to see substantial intellectual demands beyond those required at 
the middle school level. This tends not to be the case in any domain in the district’s English 
9 Framework. Overall, no clear developmental progression is evident from middle to high 
school. In fact, there are many fewer demands in grade 9 than in the previous grades. 
Admittedly, these differences stem from the way in which the corresponding documents are 
structured at the state level — a structure the district adopted. Unfortunately, in the end, the 
district indicators suffer from a lack of continuity, which undermines progression and makes 
the 9 th -grade standards less challenging than they should be. 

In math, most of the district indicators for preK-8 ramp up across grades with progression 
being strongest at the earlier grades. Geometry stands out as an exceptional strand with 
essential knowledge and skills carefully laid out from prekindergarten through the high 
school Framework. 

However, if MCPS students are to meet the expectations Achieve’ s benchmark standards set 
with respect to algebraic understandings, it would be helpful to shift some content now in 
Algebra I to 8 th grade, adjusting the content of the algebra standard in 6 th and 7 th grade as 
appropriate. This could include additional work on linear functions and linear equations and 
might also include work on nonlinear functions, operations on polynomials and the 
mathematical solution of quadratic equations — topics treated in Achieve’ s Foundations for 
Success. To strengthen the Algebra I course, MCPS would benefit from reviewing the 
Algebra I course standards developed by the state of Massachusetts, even though this 
document is not a formal benchmark for Achieve. While the two courses are roughly 
equivalent, Massachusetts places more emphasis on important concepts within number sense 
and relationships, including rational and irrational numbers, and on formal mathematical 
reasoning about real numbers. For example, Massachusetts requires Algebra I students to 
“use properties of the real number system ... to prove or disprove statements, and to justify 
every step in a sequential argument.” 

As was the case in English/language arts, the task of aligning with the state’s high school 
Core Fearning Goals in math presented some challenges because the state K-8 standards do 
not mesh readily with the high school Core Fearning Goals. However, for the most part, the 
district successfully incorporated the state goals, assessment limits and skill statements — 
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meant to define more closely the intent of the state indicators — into its course-level 
Frameworks. 

Increasing the level of rigor in the math Framework can be good in and of itself and is 
distinct from acceleration (having students complete algebra in 8 th grade or earlier). Achieve 
offers two cautions with respect to acceleration. Acceleration should not come at the expense 
of depth (having students develop conceptual understanding and learn to apply mathematics 
to new situations, even while they gain numerical and procedural fluency). Holding standards 
constant, but letting the time students are given to meet them vary, is a fundamental principle 
of the standards movement. Students do not necessarily acquire proficiency at the same 
developmental rate, and students who take Algebra I in 9 th grade are on track to take Pre- 
Calculus in grade 12. It is important not to lose sight of the overarching goal of preK-12 
math instruction stated in the introduction to the county’s Framework: “Students with 
mathematical proficiency understand basic concepts, are fluent in performing basic 
operations; reason clearly; formulate; represent and solve mathematical problems; and 
maintain a positive outlook toward mathematics.” 

• How well do the MCPS high school semester tests in English 9, Algebra I, and 

Geometry measure the knowledge and skills laid out Maryland’s Core Learning Goals? 

Are they predictive of student success on the state tests? 

Montgomery County’s semester tests are high quality and align to both the district 
Frameworks and the state tests. As a result, students who perform well on these tests are 
likely to perform well on the corresponding state tests. 

The district’s English 9 tests demand a high level of accomplishment. The reading passages 
include both fiction and nonfiction; present a range of engaging topics; and are written at an 
appropriate, yet challenging, level for grade 9 students. Because the passages are complex 
and rich, it was possible to generate worthwhile questions that assessed the skills and 
concepts described in the state’s Core Learning Goals. Test questions also are thoughtfully 
constructed. Multiple-choice items are fair, straightforward and address important aspects of 
the related passages. Constructed-response items are carefully developed and offer students 
the opportunity to generate meaningful and divergent responses that can be supported with 
evidence from the texts. Alignment of the tests to Maryland’s Core Leaning Goals is strong, 
indicating the district tests faithfully measure those skills and concepts expected by the state. 
It also is important to note that the test forms, administered at the close of the first semester, 
hold students to an appropriate level of expectation given one semester’s worth of 
instruction, and success on these forms would provide a sound basis for achievement on the 
second semester exam. 

District math tests are equally solid. By including brief and extended constructed-response 
items, along with multiple-choice items, the district telegraphs a clear message that it expects 
students to be able to reason, problem solve and communicate mathematically. Algebra IA 
and IB, and Geometry A and B semester tests align well with the content and performances 
delineated in the Maryland High School Core Learning Goals on all counts except one. The 
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second semester test for Algebra I contains a number of challenging items that extend 
beyond the content of the Core Learning Goals. These items, however, align to the district’s 
Framework, which tends to be more rigorous than the state’s. The geometry indicators are 
quite impressive, not only providing a rigorous interpretation of the state’s standards and 
learning goals, but also faring well in a comparison with Achieve’ s benchmark standards. In 
short, students completing this two-course sequence should be well prepared for more 
advanced mathematics courses. 

• How challenging are the MCPS high school tests? Are they sufficiently rigorous that 
students passing these tests will be prepared to complete more advanced courses? 

As previously mentioned, MCPS tests generally are rigorous and assess important high 
school content. However, if the goal is to have most students prepared for higher-level 
courses such as AP English and Calculus by 12 th grade, the district should raise the level of 
rigor of its English 9 “honors” level first semester test and its Algebra I and Geometry tests. 

When reviewers compared the “honors” level and the “at-level” form of the grade 9 English 
tests administered at the end of first semester, they found the level of the challenge to be 
roughly comparable. This raised questions about the level of demand of the honors exam and 
the usefulness of the data it provides schools and students. If scored student work had been 
available to enable reviewers to be sure of the standards students were held to on 
constructed-response items, they might have discerned a difference in the level of challenge 
expected of the two student populations. In the case of the end-of-the-year semester tests, 
reviewers found, as would be expected, that the level of challenge of the “honors” test was 
somewhat higher than the “at-level” test. 

Reviewers found the district’s semester tests for Algebra I and Geometry address challenging 
content, but the level of rigor of test items, taken collectively, is somewhat depressed. Even 
if “calculus for most students in grade 12” was not the goal, the district might want to review 
the level of demand of items for the effect they have on teaching and learning. Reviewers 
rated the vast majority of items either as Level 1 (recalling information or a simple 
procedure) or Level 2 (requiring decisions as to how to approach the problem or activity). 
There were few Level 3 items (requiring complex reasoning, strategic thinking, using 
evidence, explaining the method of solution or making conjectures). A number of states have 
demonstrated that it is possible to raise the level of rigor on tests administered to large 
numbers of students in standardized settings by including demanding, Level 3 selected- 
response and constructed-response items. 

* * * 

In sum, MCPS has developed high-quality Curriculum Frameworks and semester tests in 
English/language arts and math. The Frameworks do an effective job articulating the 
county’s academic expectations, and those expectations accurately reflect and expand upon 
Maryland’s standards. While the MCPS expectations overall are quite challenging, they 
could be strengthened in a few specific areas noted by Achieve. The county’s semester tests 
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in English 9, Algebra I and Geometry are thoughtfully constructed and well aligned to both 
the county’s Frameworks and the state’s Content Standards and Core Learning Goals. This 
should make them good predictors of student performance on the state’s end-of-course 
exams, which students will soon need to pass to graduate. As with the Frameworks, however, 
Achieve has recommended several ways to improve the exams and make them more 
challenging. This is particularly important in light of the county’s ambitious goal that the 
majority of its students be enrolled in college-level courses by 12 th grade. 

In the course of completing our analysis, it became evident that MCPS has made remarkable 
progress in creating and continually improving an aligned system of Frameworks, 
instructional guides, diagnostic tests and other instructional supports. There is more work to 
be done, for sure, but this careful attention to high standards and capacity should go a long 
way toward helping teachers and students understand and meet the intellectual demands of 
the 21 st century. 
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Major Findings: English/Language Arts Frameworks 



The MCPS Curriculum Frameworks for English/language arts are organized by grade level for 
grades preK-8 and by course at the high school level. Although Achieve’ s review was limited to 
the Curriculum Frameworks, we did have occasion to refer to the supplementary Instructional 
Guides in completing our analysis and were impressed with the level of support these provide for 
teachers. In some instances, they contain detail that could be drawn upon to make the 
Frameworks even clearer. 

The following summary highlights and explains the most important findings from Achieve’ s 
benchmarking study of the MCPS Frameworks and their relationship to Maryland’ s Content 
Standards and Core Feaming Goals. 

Strengths 

• The MCPS preK-8 and grade 9 Frameworks in English/language arts are comprehensive. 

Key content and skills in Reading, Writing, English Fanguage Conventions, Fistening and 
Speaking are all contained in the Frameworks. There is no fundamental area of English/language 
arts that the documents overlook. The district clearly paid close attention to the Maryland 
English/Fanguage Arts Content Standards and used them as the foundation for their preK-8 
grade-level indicators, adding detail and sometimes expanding them. By more fully developing 
the following areas, the district can fill in gaps and make its expectations more challenging and 
explicit. 

• Phonemic Awareness, Decoding and Word Recognition. While explicit, systematic 
phonics study is included in the Framework, it is no more detailed than the state 
standards. The expectations for phonemic awareness at kindergarten are not as 
comprehensive or demanding as, for instance, California’s. With the new push for early 
literacy, Montgomery County should think about increasing the demands and making the 
indicators more explicit to ensure all students have the skills they need to succeed. 

• Literature Study. The Frameworks do not provide detailed guidelines as to the type of 
analysis students should be able to conduct as they move into high school. 

• Nonfiction or Informational Text. The Frameworks could be strengthened by specifying 
the types and/or range of materials students should know how to read and comprehend, 
e.g., newspapers, instructional manuals, warranties, policy statements, speeches and 
various workplace documents. 
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• One of the greatest strengths of the MCPS indicators is their use of clarifying examples. 

The thumbnail sketches provided in the English/language arts Frameworks at once provide 
curriculum development guidance to teachers and a window for parents into classroom practice, 
as the following examples show: 



CHARACTERISTICS OF LITERARY GENRES CLARIFYING EXAMPLES 


GRADE 1 


Identify elements of a poem such as rhyming words, repeated sound and 
rhythmic patterns. 


GRADE 2 


Compare a poem to a story about the same topic (e.g. spring), and discuss 
differences between the two types of literature. 


GRADE 3 


Contrast the characteristics of a poem, play and a story. 



8 th -GRADE WRITING CLARIFYING EXAMPLES 

Write an opinion piece for a school or community publication that addresses an issue 
with focus and uses transitions and supporting details effectively. 

Write a short story that includes figurative language and foreshadowing. 

Write a resume and cover letter for a position in your school government. 

Write an essay or report which is organized around a specific rhetorical structure. 
Select a form for a piece of writing and spend an appropriate amount of time planning, 
drafting and editing the piece. 



Reviewers urge the district to include clarifying examples at prekindergarten and provide 
multiple examples wherever feasible at other grades. In addition, the district may want to refine 
some clarifying examples because they are not always on target. Some examples seem to be 
mere restatements of an indicator, while other examples are simply not as good a match as they 
could be. 

Many of the reading standards repeat grade to grade, which makes it especially critical that the 
complexity and sophistication of what students are being asked to read is defined with precision. 
Moreover, inclusion of titles from the district’s instructional guides would help to make the 
clarifying examples even more concrete. The district grade 7 standard 2.7.3, “Discuss the 
portrayal of historical events in two or more texts (e.g., Anne Frank: The Diary of a Young Girl 
and Zlata’s Diary: A Child’s Life in Sarajevo ) and compare how the characters confront similar 
challenges and situations,” is a good start and should be emulated throughout the 
English/language arts indicators. 

• The format of the preK-8 document is organized and clear, and the indicators generally 
are concise and are jargon-free. 

Overall, the organizational structure of the preK-8 Framework is sensible. The connections to 
the Maryland standards are consistent and clear. All too often, standards documents are only 
comprehensible to readers familiar with “standards speak.” Refreshingly, Montgomery County 
indicators are written succinctly, without becoming so cryptic that only educators can understand 
what is meant. These standards should be especially clear for an educator audience and should 
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prove useful in informing curricular planning. To make the document even more understandable 
to parents and the general public, the district might want to consider defining some terms such as 
“phonemic awareness,” “consonant blends,” “alphabetic principle,” “cueing systems” and 
semantics.” 

Areas for Improvement 

V Progression is uneven. Of special concern is the disconnect between the organization 

and content of the preK-8 Framework and grade 9 Framework. 

In some areas, the district successfully augmented the state standards by clarifying or 
strengthening the development of content over time. However, in other areas, to make district 
indicators comparable to the benchmark standards, the district should consider introducing key 
ideas earlier or developing indicators more completely. Specifically: 

• Making and Revising Predictions should continue to be introduced in preK and then 
carried forward through the elementary grades as it is in the benchmark documents. 

• Understanding Cause and Effect could be described in more instructionally relevant 
terms in order to clarify the increasing demands through the primary grades. 

• Analysis of Literary Text would be more robust if figurative language was introduced 
earlier and treated more thoroughly. 

• Vocabulary Acquisition would be strengthened if more attention were given to conceptual 
understanding. 

• Writing Voice and Style should be fleshed out to include more direction about how 
students develop a voice, tone and style of writing in the early grades. 

• Spelling should include indicators that address what teachers should expect of students in 
the early grades in terms of their spelling development. 

• Comprehension strategies should emphasize the key idea that, before such strategies are 
of any real use, the students must be aware of self-monitoring their understanding and 
becoming aware of when the text is not making sense. 

• Cause and Effect, and Fact and Opinion would be treated more effectively if the 
introduction of fact and opinion were postponed and treated separately from cause and 
effect. 



Since the county developed its grade-by-grade indicators directly from the state’s indicators at 
benchmark grades, the MCPS preK-8 Framework corresponds closely to the Maryland State 
Content Standards. Similarly, the district bases its grade 9 Framework, with some additions, on 
the state’s High School Core Learning Goals and Assessment. MCPS limits and further 
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organizes the content around the themes of choices, conflicts, changes and relationships. It is 
unfortunate that the formats of the preK-8 and high school state documents are fundamentally 
different and require readers to reorient themselves. Not surprisingly, the corresponding MCPS 
documents follow suit. 

The district might want to rethink its decision to maintain high school learning goals that do not 
appear to progress from the preK-8 standards. Although Maryland’s High School Core Learning 
Goals in English/language arts do not carry over many significant topics from the state standards 
document at grade 8, there is no compelling reason for the district to perpetuate this practice. 

To be specific, Maryland’s High School Core Learning Goals do not address many of the 
indicators within the following strands: 

• Structural Features of Text; 

• Comprehension and Interpretation of Informational Text; 

• Evaluation of Informational Text; 

• Independent Reading; 

• Comparison of Literary Text From Diverse Cultures; 

• Practical Writing; and 

• Acquisition and Application of New Vocabulary. 

Additionally, because the wording is so different between Maryland’s Content Standards and its 
Core Learning Goals, even those elements that are included in both (for example, writing to 
inform or persuade) are described as having different characteristics. This makes using a similar 
writing rubric in grades 8 and 9 problematic. The result might be an unfortunate disjuncture 
between how grade 8 teachers talk to their students about writing and how grade 9 teachers talk 
to their students. Having preK-12 standards that flow in a continuous stream of important 
content and skills will eliminate this kind of troublesome disjuncture between grade levels. 

A related concern, which also springs from the district’s decision to replicate the organization 
and structure of the state’s Core Learning Goals, is that topics contained in the English 9 
Framework in many instances are addressed only in summary terms, as the following examples 
illustrate: 

• Vocabulary Development. Grade 9 mentions interpreting figurative language and a 
vocabulary list, but goes no further. 

• Knowledge of Various Literary Genre and Forms. The preK-8 standards take note of 
differences among prose, poetry and drama, and make some finer distinctions, but those 
drop off in grade 9. For example, different forms of poetry and different forms of 
imaginative literature (myths, fables, tall tales, fairy tales) are only mentioned in the K-8 
document. 
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• Literary Criticism. The mention of literary criticism in grade 9 is underdeveloped (1. 1.3.2 
only states “students will use a variety of critical approaches [traditional, mythological, 
etc.] to analyze text and synthesize ideas.”). 

• Literature Study. While the standards cover the basics, there is not great depth or 
specificity with respect to the type of analysis students should be able to conduct as they 
move into high school. 

• Nonfiction or Information Text. The grade 9 Framework does not reference the types 
and/or range of materials students should know how to read and comprehend (e.g., 
newspapers, instructional manuals, warranties, policy statements, speeches and various 
workplace documents). In grades 4-8, the indicators include a nice variety of different 
cognitive demands. But by high school, there are no additional demands regarding 
evaluating texts for patterns of organization, syntax or word choice; for credibility; or for 
logical, deceptive or faulty modes of persuasion. In fact, informational and expository 
texts do not seem to be included in the grade 9 course of study. 

• Research Skills. In grade 9, research skills creep in under 2.2.5, but they are not handled 
comprehensively. 

An exception to the pattern described above is the treatment given to the standards concerned 
with “English Language Conventions” and “Qualitative Aspects of Essay Writing.” Both contain 
substantial, well-described content. 

C The logic behind what content was included in which unit is not clear in the grade 9 

Framework. 

Reviewers found it difficult to determine the rules governing the selection of content for each 
unit as the following examples explain: 

• Unit 1. Indicator 2.3.1, “The student will identify sources of information on a self- 
selected and/or given topic,” and 2.3.2, “The student will use a systematic process for 
recording, documenting and organizing information,” are both included. However, the 
only kinds of writing that students are expected to do in Unit 1 are descriptive and 
personal. It is not clear for what ends students are identifying resources, and recording 
and organizing information. One would expect writers to engage in these tasks when they 
were going to write informative or persuasive writing, but Indicators 2.1.1 and 2.1.4 are 
not included. 

• Unit 2. Students are asked to meet 2.2.1 (prewriting) and 2.2.3 (revision) but they are not 
asked to produce any writing at all; 2.1.1 through 2.1.4 are not included in Unit 2. It is 
unclear what they are prewriting and revising. This also is the case in Units 3 and 4. 
Students are engaging in various stages of the writing process, but Indicators 2.1.1 
through 2.1.4 are not included. 
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• Units 3 and 4. As MCPS units are thematic, one would expect that students would 
discuss the thematic connections between what they are reading in every unit. 
Consequently, it seems 1.3.5, “The student will explain how common and universal 
experiences serve as the source of literary themes which cross time and cultures,” should 
be included in Units 3 and 4, and not just in Units 1 and 2. 

• Unit 1. Indicator 1.1.1 (pre-reading strategies) is followed by Indicator 1.1. 1.1, “The 
student will use metacognitive strategies (re-reading, using context clues to understand 
difficult words, predicting-and-supporting) to comprehend text.” The strategies included 
here overlap with the during-reading strategies included in 1.1.2. Therefore, it seems 
1.1.2 should be included in Unit 1. It also was not clear why during-reading strategies 
should be listed as an additional indicator following pre-reading strategies. 

• Unit 2. Indicator 2.1.2 (compose to describe) is followed by 2. 1.2.1, “The student will 
demonstrate skill in using the writing process — prewriting, organizing ideas, drafting, 
revising, editing and polishing — to craft organized, coherent responses to literature or to 
express personal ideas.” The numbering suggests that this indicator will somehow be an 
elaboration upon the state’s Indicator 2.1.2. However, the district indicator encompasses 
state expectation 2.2, which reads, “The student will compose texts using the prewriting, 
drafting and revision strategies of effective writers and speakers.” The placement of 
district Indicator 2. 1.2.1 makes its intent unclear and may inadvertently obscure the point 
the district is trying to emphasize to its teachers. 

• The same problem appears with county Indicator 2.2.3. 1 “The student will review rules 
governing grammar, mechanics and language usage in order to improve effectiveness in 
writing and speaking.” This would seem to fit well with the state Indicator 3.1, “The 
student will demonstrate understanding of the nature and structure of language, including 
grammar concepts and skills, to strengthen control of oral and written language.” 
However, 2.2.3. 1 appears under 2.2.3, “The student will revise texts for clarity, 
completeness and effectiveness,” which focuses not on grammar as much as organization, 
clarity and sentence structures. 

By limiting the number of indicators in each unit, MCPS unnecessarily limits the instruction 
within each of the units. A unit could address many more indicators than are listed here because 
one hour of instruction might touch on multiple indicators. This would not be the first year that 
students have been introduced to the content within the indicators; most of the indicators touch 
on issues that students have been working with for many years (such as the pre-, during- and 
after-reading strategies; the elements of literature; the writing process; various writing genres, 
etc.). Each unit at grade 9 could and should go deeper than it does. 

It would probably be more helpful for teachers to see how an indicator might build in demand 
during the course of the year. Perhaps the district could develop some kind of a progression to 
address this issue. As a case in point, Indicator 1.3.5 reads, “the student will explain how 
common and universal experiences serve as the source of literary themes which cross time and 
cultures.” Teachers could be given guidance for developing this indicator across units. For 
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example, in Unit 1, students could compare their own experiences to the theme. In Unit 2, 
students could compare the theme to the experiences of themselves and others. In Unit 3, 
students could compare the theme to the themes in other novels they have read or films they have 
seen. In Unit 4, students could analyze and discuss themes that cross time and cultures (such as 
good vs. evil) and compare those to themes that may be unique to American culture (such as the 
idea of the Western frontier or America as a land of opportunity). 

•S The proposed “progression” across grade 9 units is sometimes puzzling. 

Unit 1, for example, starts with students using pre-reading strategies. (1.1.1 “The student will use 
pre-reading strategies appropriate to both the text and purpose for reading by surveying the text, 
accessing prior knowledge, formulating questions, setting purpose(s), and making predictions.”) 
During-reading and after-reading strategies are not included in Unit 1. 

Unit 2 starts with students using after-reading strategies. (1.1.3, “The student will use after- 
reading strategies appropriate to both the text and purpose for reading by summarizing, 
comparing, contrasting, synthesizing, drawing conclusions and validating the purpose for 
reading.”) Pre-reading and during-reading strategies are not included in Unit 2. 

Unit 3 starts with students using during-reading strategies. (1.1.2, “The student will use during- 
reading strategies appropriate to both the text and purpose for reading by visualizing; making 
connections; and using fix-up strategies such as rereading, questions and summarizing.”) Pre- 
reading and after-reading strategies are not included in Unit 3. 

This proposed “progression” of instruction across these units would benefit from the latest 
research on effective instruction in English/language arts. When introducing students to a new 
topic or text, effective teachers will engage in activities to support pre-, during- and after-reading 
through activities l ik e “Know, Want to Know and Learn,” which help students build on the 
knowledge they already have and focus on new learning, or the following: 

1. Engage in a pre-reading activity designed to activate students’ schema and orient them 
to the lesson. 

2. Model effective during-reading strategies and provide support to readers who are 
having trouble comprehending (perhaps in the form of think-along questions or 
discussion questions). 

3. Engage in after-reading discussion and activities designed to reinforce what students 
have learned and encourage reflection and metacognition. 

The way that the Units are organized within the grade 9 Framework suggests that readers are not 
always given support in using effective strategies before, during and after reading. 
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■S The MCPS indicators that were added throughout the Frameworks sometimes are 
unclear. 

Reviewers were not always sure how to interpret indicators, as the following two examples 
illustrate. 

• Indicator 1.2. 1.1 states, “The student will use all the components of a text to draw valid 
conclusions about the text and its meaning.” The phrase “all the components” is not clear. 
Does this refer to the literary elements of a fictional text (plot, character, setting)? Does it 
refer to the stylistic decisions, word choice and tone of the author? Or does it refer to the 
organization and format of a nonfiction text? If it refers to all of the above, then it needs 
to be stated more explicitly to be of use to educators planning instruction. 

• Indicator 1.2. 1.2 says that, “the student will demonstrate skill in analyzing the 
components of plot structure and synthesizing his ideas into an organized, coherent piece 
of writing.” The state indicator above this district-level indicator (1.2.1) addresses the 
consideration of plot, character, setting, conflict and point of view. It was not clear to 
reviewers why the district focuses only on plot in its indicator. It would have made sense 
if, in future units, educators were expected to focus more directly on other elements, such 
as character or setting, but in fact, none of the other units include a county indicator for 
1 . 2 . 1 . 

Reviewing grade 9 indicators to ensure their meaning is as transparent as possible will support 
effective instructional planning. 
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Major Findings: Grade 9 Semester Tests in 
English/Language Arts 



Achieve carried out a detailed study of the rigor of the MCPS tests in English 9 and their 
alignment to the Maryland Core Learning Goals. 

Achieve reviewed four English tests: 

• English 9A Assessment, Form 1, January 2002 (at-level); 

• English 9A Assessment, Form 2, January 2002 (honors); 

• English 9B Assessment, Form 1, June 2002 (at-level); and 

• English 9B Assessment, Form 2, June 2002 (honors). 

The tests are designed to assess the MCPS English 9 Framework (May 2002), which is based on 
the Maryland Core Learning Goals and “assessment limits,” statements that further explain the 
testing boundaries for each goal for grades 9-12. The MCPS English 9 Framework aligns the 9 th 
grade English course with the expectations defined in Maryland’s Core Learning Goals, 
expectations and indicators; the Maryland Content Standards; and the “trajectory of preparation 
for Advanced Placement (AP) courses in English Language and Composition, and English 
Literature and Composition.” 

The county’s document for grade 9 includes those expectations from the state that are designated 
to be assessed at grade 9, and identifies certain state assessment limits to be addressed within 
each quarter of instruction for the year. Each quarterly unit centers on enduring understandings 
and essential questions focused on a theme: choices, conflict, communication and change. While 
some indicators and expectations are common to all four units, others are specific to one 
thematic unit. Additionally, the scope and sequence includes Maryland State Department of 
Education assessment limits that define the content of the English I Maryland High School 
Assessment (English 9). The county’s semester tests, then, target the assessment limits for two 
units’ instruction. 

Each test is composed of a number of selected-response items, two brief constructed-response 
items, and one extended-response item. The English 9A tests have 40 selected-response items 
and the English 9B tests have 50 selected response items. The two forms of English 9A share a 
common subset of items, while the English 9B forms are discrete. Each of the tests was analyzed 
using Achieve’ s protocol. 

Reviewers were concerned with answering four questions: 

1. Do both forms address the content and performances cited in the Core Learning Goals as 
explicated by the district English grade 9 Framework? 
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2. Are both forms of the test appropriately constructed to return meaningful information to 
the students and schools relative to their performance in achieving the Core Learning 
Goals? 

3. Is the level of rigor different between forms 1 and 2, with the “honors” level (form 2) 
being more demanding than the “at-level” test (form 1)? 

4. Are the tests an appropriate measure of the skills described by the Maryland Core 
Learning Goals as expected of grade 9 students after one semester’s worth of instruction 
or after one year’s instruction? 

Strengths 

• MCPS grade 9 semester tests in English/language arts are well aligned to the Maryland 
Core Learning Goals and to the district’s grade 9 Framework. 

Grade 9 test items on all test forms strongly match both the content and performances set forth in 
the Maryland Core Learning Goals and the district Framework. The content match across all 
forms was quite high, ranging from 94 percent to 98 percent. Achieve has reviewed more than 30 
large-scale English/language arts tests and these results are among the best we have seen. 
Although the alignment of test items to the performances described in the Maryland Core 
Learning Goals was slightly lower, ranging from 86 percent to 96 percent across all forms, the 
match still is quite strong. In addition, all forms of the tests were well balanced in their attention 
to reading, writing and language skills, and all forms attempted to mirror the balance of skills 
suggested by the state’s Core Learning Goals. All in all, these results provide ample evidence 
that district tests successfully measure the content and performance expectations described in the 
state goals. 

• In general, the district’s grade 9 semester tests in English/language arts are rigorous 
assessments of the Maryland Core Learning Goals and of the district’s Frameworks. 

The district semester tests, both the “at-level” and “honors” level forms, are high quality and 
demand a relatively high level of literacy appropriate for 9 th grade. They are consistent with the 
level of challenge described by the Maryland Core Learning Goals. 

The tests are characterized by having substantive reading passages, written at an appropriate yet 
challenging level for grade 9 students. The range of passages sample fiction and nonfiction 
genres, and present a variety of topics that students would find engaging. In fact, Achieve found 
the set of reading passages on each form to compare favorably with those state tests that had 
sound reading assessments. By selecting passages for inclusion on the tests that are relatively 
complex and permit multiple interpretations, the district positioned itself to generate worthwhile 
multiple-choice and constructed-response questions that assessed the concepts and skills 
contained in the Maryland Core Learning Goals. The semester tests are based on the state 
standards, which are robust, and a student’s overall performance on these county tests should 
prove to be a reliable predictor of a student’s performance on the state test. 
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• The grade 9 semester tests in English/language arts contain a well-considered mix of item 
formats. 

All forms of the tests are constructed with a combination of item formats, including multiple- 
choice (selected-response) items as well as both brief and extended constructed-response items. 
For the most part, selected-response items were fair, straightforward and relevant to the texts on 
which they were based. Constructed-response questions, where a student has to think through 
and develop a response from scratch, asked students to develop meaningful and divergent 
responses able to be supported by evidence from the texts. 

Areas for Improvement 

S The district should adjust the level of rigor of its first semester honors-level test in 
English/language arts and screen items more closely for technical flaws. 

It was difficult to discern a difference in rigor between the “honors” and “at-level” form of the 
first semester test. One of the ways reviewers judge the overall rigor of a test is by tracking the 
level of demand — an alignment criterion that gauges the kind of thinking an item requires of a 
student. Forms 1 and 2 seem to be pretty evenly divided between items at Level 1 — basic recall 
and comprehension, and Level 2 — inference and procedural application. In addition, the reading 
demands of both forms of the test were comparable, with neither form calling for a higher degree 
of reading ability than the other. The only real difference might lie in the nature of the extended 
constructed-response questions on the two forms. Arguably, the topic on the honors-level test is 
of a more challenging nature than that on the at-level test. However, in the absence of scored 
sample papers, reviewers had no way of discerning whether that was, in fact, the case. In sum, if 
the goal was to construct first semester forms that represented differing levels of challenge for 
the two groups of students, these forms did not seem to accomplish that goal. 

Judging the level of demand of the two forms of the test administered at the end of the year was 
also not a straightforward task, but for a different reason. The honors-level test contained a 
significant number of multiple-choice items that reviewers eliminated from consideration due to 
problems in the way the items were constructed. Common examples of technical flaws in 
English/language arts items include having no or multiple correct answers; misleading graphics; 
or reading items that are “text independent,” meaning the student can answer the item correctly 
without having to read the passage upon which the item is based. When an item is technically 
flawed, it increases the likelihood of false positives, where students get the right answer for the 
wrong reason, or false negatives, where students get the wrong answer but may have the 
knowledge to answer the item correctly. Defective items may cause the district to draw the 
wrong conclusions about which concepts or skills are causing students difficulty. 

In the case of the end-of-the-year honors-level test, most of the multiple-choice items judged to 
be flawed contained multiple defensible answers. Some of these items, if constructed without 
flaw, might have been evaluated as Level 3 items — questions requiring students to form an 
interpretation and then select an answer that best matched that view. Had reviewers been able to 
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include these flawed items, they might have concluded the overall level of rigor of the multiple- 
choice items on the honors test was indeed higher. Reviewers, however, did find some 
differences in the level of challenge on the two forms of the end-of-the year test in the number 
and kinds of reading passages included and in the level of demand of the constructed-response 
items. These comparisons led the reviewers to conclude the end-of-the year honors-level test was 
somewhat more rigorous than the corresponding at-level test. 

While the number of questionable items certainly is not high enough to nullify student results, 
the district may wish to consider field testing items for the grade 9 tests with some of its grade 10 
students or vetting the items by having a number of professionals examine the items for potential 
flaws. Students and teachers need to have faith in the test results. That said, the district has made 
considerable strides in measuring demanding content and skills in a two- semester sequence and 
the tests can be readily improved over time. 
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Major Findings: Mathematics Frameworks 



Achieve reviewed the county’s Curriculum Frameworks in mathematics, comparing it to 
Maryland’s Core Learning Goals and exemplary standards from other states and nations. 

Although Achieve’s review was limited to the Curriculum Frameworks, we did have occasion to 
refer to the supplementary Instructional Guides in completing our analysis and were impressed 
with the level of support these provide for teachers. In some instances, they contain detail that 
could be drawn upon to make the Frameworks even clearer. 

The MCPS Curriculum Frameworks for Mathematics, last reviewed and revised in May and June 
of 2002, are organized by grade level for grades preK-8 and by course (Algebra I and Geometry) 
at the high school level. The Curriculum Framework for grades preK-8 is organized into 10 
standards, with standards 1 through 6 devoted to mathematics content and standards 7 through 10 
devoted to the mathematical processes. The six content standards are: 

• Algebra, Patterns and Functions, 

• Geometry, 

• Measurement, 

• Statistics, 

• Probability, and 

• Number Relationships and Computation. 

The four process standards for grades preK through 8 mirror the mathematical process standards 
adopted by the state of Maryland in their Mathematics Content Standards (May 2000). These 
standards are not differentiated by grade level and are defined for the following processes: 

• Problem Solving, 

• Communication, 

• Reasoning, and 

• Connections. 

The MCPS High School Curriculum Frameworks for Algebra I and Geometry were developed to 
align with the Maryland Mathematics Content Standards and the Maryland High School Core 
Learning Goals (March 2001). The latter document provides the foundation and organizing 
structure for the scope and sequence of indicators that MCPS has defined for Algebra I and 
Geometry. Maryland High School Core Learning Goals 1 and 3 — Functions and Algebra, and 
Data Analysis and Probability — are the basis for the Algebra I course while Goal 2 — 

Geometry, Measurement and Reasoning — is the basis for the Geometry course. 

Achieve used a cadre of mathematics educators and mathematicians to review and benchmark 
the MCPS Curriculum Frameworks for mathematics. The reviewers were selected because of 
their deep knowledge and experience with mathematics, mathematics education and preK-12 
mathematics standards — and because they represent a range of perspectives relative to 
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mathematics that, when taken collectively, provide MCPS with a well-rounded, well-informed 
perspective on their standards. The charge to the reviewers was to critique the Curriculum 
Frameworks using a set of guiding questions drafted by Achieve and compare the district’s 
standards to other mathematics standards identified as exemplary. Achieve’ s benchmark 
documents for mathematics include the standards from Japan, Arizona and Achieve’ s 
Foundations for Success. The MCPS Curriculum Frameworks also were compared to the 
Maryland Content Standards for the end of grades 3, 5 and 8 and to the Maryland High School 
Core Learning Goals. 

For purposes of this analysis, it is important to place Japan’s standards and Achieve’ s, 
Foundations for Success — limited to the middle grades — in the larger context of mathematics 
education reform across the nation. Achieve ’s expectations look to the future; no state has yet 
articulated standards of equal rigor, and no state or district is yet fully prepared to implement and 
hold students accountable for what is admittedly a high level of achievement. For this very 
reason, Achieve’ s Mathematics Achievement Partnership (MAP) with 14 states entails more than 
formulating a set of rigorous, end-of-grade 8 standards. To support implementation of these 
demanding standards, Achieve also is generating prototypes of effective professional 
development, back-mapping standards for grades 3-7 and reviewing curriculum materials to 
determine their degree of alignment with Foundations for Success. Achieve’ s benchmarking 
process is designed to compare a set of state or local standards to powerful exemplars, such as 
Foundations for Success and Japan. It is therefore unrealistic to expect a state or district set of 
standards — not to mention teachers and students — to align with these external benchmarks 
overnight. The message has been and remains one of “continuous improvement.” 

The guiding questions that framed the expert reviews of the MCPS Curriculum Frameworks for 
mathematics were organized according to criteria that Achieve and others have determined to be 
qualities that distinguish exemplary state standards from marginal ones. A key criterion is rigor, 
and Achieve’ s reviewers critiqued the county’s Frameworks for rigor from a variety of 
perspectives — the extent to which key aspects of mathematics were included in the standards, 
the appropriate grade placement of content and process skills, and progressive demands across 
grade levels. Reviewers also evaluated the MCPS Curriculum Frameworks for focus, clarity and 
specificity. These criteria were used as the basis for identifying and explicating the strengths of 
the MCPS Curriculum Frameworks as well as the challenges the district likely will face as it 
moves forward in implementing its student expectations. 

The following summary highlights and explains the most important findings from Achieve’ s 
benchmarking study of the MCPS Frameworks in Mathematics and their relationship to 
Maryland’s Content Standards and Core Learning Goals. 



Measuring Up — Montgomery County 



36 



Achieve, Inc., 2003 





Strengths 



• The MCPS Curriculum Frameworks for Mathematics are quite consistent with 
Maryland’s Content Standards and High School Learning Goals. 

MCPS indicators closely parallel the content standards that Maryland provides for grades 3, 5, 8 
and 12, and also the state’s High School Core Learning Goals. Most MCPS indicators are 
restatements of the Maryland standards with detail and, in some instances, extension added. The 
county has done an excellent job of aligning its Mathematics Curriculum Frameworks to the 
Maryland State Mathematics Content Standards and the High School Core Learning Goals. The 
expectations of the state standards almost always are met or exceeded by the grade-by-grade and 
course-level indicators set forth in the MCPS Curriculum Frameworks. 

• The mathematics Frameworks typically contain clear, specific and measurable language. 

The MCPS indicators are clear and succinct in their statement, and they are as jargon-free as that 
clarity and succinctness will allow. Because of this clarity, and also because of the degree of 
specificity that has been added by MCPS as they have extended and expanded the state’s 
expectations, the mathematics Frameworks are quite accessible to the general public. In addition, 
the format, numbering schema and glossary — constructed with a lay audience in mind — 
contribute to the document being “user friendly.” MCPS indicators are not as fine grained as the 
expectations in the Arizona standards document, but this is not a deficit. Rather, it is an asset in 
that readers of the document will tend not to be overwhelmed by detail. All in all, the Framework 
document is an effective blend of readability and utility for guiding instruction. 

Another clarifying tool embedded within the Curriculum Frameworks is a series of sample 
problems, intended to help teachers and parents understand what the indicators really mean and 
understand the level of performance standards demand of students. MCPS should consider 
expanding its repertoire of such problems in its Instructional Guides in the future. One way is to 
include substantive examples, with associated commentary that illustrates more fully and 
elaborates upon the MCPS expectations — similar to what Achieve has done in its Foundations for 
Success. These illustrations would be especially powerful if at least some of the added problems 
were to cut across the traditional domains of mathematics, such as geometry and measurement. 

The Program for International Student Assessment of the Organization for Economic Cooperation 
and Development focuses on this type of problem, requiring students to adapt and combine 
methods from several of the separate content domains. 

• The MCPS Curriculum Frameworks for Mathematics are comprehensive. 

All the standard domains of traditional school mathematics are represented in the MCPS 
indicators: the six content strands of Algebra, Patterns and Functions; Geometry; Measurement; 
Statistics; Probability; and Number Relationships and Computation; and also the four mathematical 
processes of Problem Solving, Communication, Reasoning and Connections. The grade-by-grade 
indicators and course-level indicators do a good job of providing students with a strong foundation 
in mathematics. Moreover, most of the content in the MCPS Framework also appears in the 
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benchmark documents, meaning that MCPS has not included “extraneous” expectations. Given its 
intent to address all of the state’s High School Core Learning Goals in a two-course sequence, 
MCPS also does a good job of weaving the significant ideas of data analysis and probability into a 
fairly traditional high school mathematics curriculum. Probability is covered in a traditional 
counting way, but it is addressed for its part as the cornerstone of statistical inference. 

• In general, the Curriculum Frameworks for Mathematics lay out a well-considered 
progression of expectations from preK through the high school courses. 

In general, the MCPS indicators show a thoughtful progression of knowledge and skills across the 
grade levels and at the secondary level from course to course. Such articulation enhances the 
l ik elihood that students will be able to build on prior knowledge and experience and hence attain 
the high expectations set forth in the Curriculum Frameworks. The district deserves credit for the 
work it has done to break the state Content Standards and the High School Core Learning Goals 
into reasonable grade-level and course-level standards. Reviewers were particularly impressed by 
the strong progression evident in the indicators related to algebra and to statistics. A good example 
is the careful development of the use of variables in equations and formulas in grades 4 through 7 
(Indicators 1.4.3. 1, 1.5.3. 1, 1.6.3. 1^4 and 1 .7.3.3 — 4). Another is the thoughtful attention given to 
arithmetic properties that are introduced in grade 4, embellished in grade 5, and then strengthened 
in grade 7 with the inclusion of expressions, and in grade 8 by combining like terms. In addition, 
reviewers pointed out that the geometry strand was particularly well laid out from preK up to and 
including high school geometry. 

While progression for the most part is strong, reviewers did point out selected instances of which 
MCPS should be aware. The measurement standard indicators require more specificity and the 
clarifying examples are in need of careful attention. As pointed out earlier, there also was minimal 
progression identified from the 8 th -grade probability indicators to those in Algebra I and also little 
development of the ideas of permutations, combinations, sample space, dependence and 
independence from grade 8 (Indicators 5.8.1, 5.8.2 and 5.8.3) to Algebra I (Indicator 3.1.3). 

In addition, while MCPS understandably adopted the form and substance of the state’s process 
standards for preK-8, the district should consider ways to help teachers understand how these 
processes “play out” at different grade levels. For example, the level of sophistication of a 
student’s repertoire of problem-solving strategies should be different in high school from what it is 
in the early grades of elementary school. The clear development of formal reasoning skills across 
the grades also is critically important, but not articulated in the MCPS Frameworks. 

• The MCPS Frameworks for Mathematics are rigorous expansions of Maryland’s standards 
and goals. 

In some instances, the Framework for grades preK-8 articulates higher expectations for students 
than do the Maryland Content Standards. These include: (1) placing more emphasis on geometry, 
especially three-dimensional geometry and transformations; (2) requiring students to achieve 
proficiency in computation with whole numbers, fractions and decimals sooner and faster, 
culminating in real number arithmetic in grade 8; and (3) expecting earlier sophistication in data 
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analysis. Reviewers, however, cautioned that the rigor of the MCPS indicators at the earliest grades 

— particularly at preK and kindergarten — might be unrealistically difficult if viewed as hurdles 
that need to be met each year. Students enter school at varying levels of readiness and display 
significant variability in their developmental levels, sometimes through 3 rd grade or so. Given the 
high standards that MCPS has set for its children at every grade — beginning in preK — 
consideration needs to be taken as to how best to deal with this variation in children’s 
developmental levels in the early elementary years. Extra support and some “give” in the grade-by- 
grade indicators may be warranted, at least until students reach a point where developmental issues 
have generally leveled out. For example, mastering Indicator 6.PK.1, where it is assumed that 
students can count to 10, model sets of 10 and read numbers to 10 in addition to understanding first 
and second may not be realistic expectations for all students entering kindergarten. Kindergarteners 
who are not yet proficient in these areas will be hindered in their ability to work with numbers to 
100, fractions of halves and quarters, and ordinal numbers up to tenth by the end of 1 st grade. 
Careful diagnostic and prescriptive work, coupled with sustained support, will be necessary to help 
youngsters acquire both the procedural and conceptual understandings that students need for 
continued success in mathematics. 

Reviewers found the MCPS Algebra I and Geometry indicators meet and sometimes exceed the 
state’s expectations, though they could not always be certain of the latter because of the way in 
which the state document is constructed. For example, Indicator 1. 1.3.1 from the MCPS Algebra I 
indicators is intended to be an extension of state Indicator 1.1.3, which requires students to apply 
the addition, subtraction, multiplication and/or division of algebraic expressions to solve problems. 
While the state is not explicit with respect to the inclusion of operations with exponents — 
although one would expect this to be a part of the expectation — MCPS clearly states in Indicator 
1.1. 3.1 that simplification of expressions using the law of exponents is part of this indicator. This 
kind of clarity at the district level helps ensure that interpretation of the state indicator will not be 
left to the interpretation of individual teachers and articulates a district indicator that clearly meets 

— and perhaps exceeds — state expectations. In another instance from Algebra I, MCPS defines 
the expectation that students understand functional transformations with respect to a variety of 
types of functions. This topic is not specifically included in the state’s expectations, and MCPS 
deserves credit for identifying and filling this gap effectively. 

Other strengths of the MCPS Algebra I indicators in comparison to High School Core Fearning 
Goal 1 are as follows: the inclusion of the exponential function, the expectation that students solve 
quadratic equations by various methods, and the requirement that students represent exponential 
and quadratic functions graphically and describe the effect of parameter changes on these 
functions. MCPS addresses the state’s expectations related to data analysis and probability (High 
School Core Fearning Goal 3) in Algebra I. The course indicators meet state expectations, but 
appear to exceed them on only one count — inclusion of the exponential curve of best fit. 

MCPS Geometry course indicators are more explicit about the reasoning tools and techniques 
required — including indirect proof and the converse and contrapositive of conditional statements 
indicators — than those defined by the state in High School Core Fearning Goal 2. The district also 
has added some important requirements on visualizing and describing loci in three dimensions. 
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Areas for Improvement 



S Although the MCPS Frameworks generally are rigorous, they do not always match the 

rigor of Achieve’s benchmarks. 

In comparing the county Framework to the benchmark standards, reviewers raised an 
overarching concern. In expanding upon the state’s standards, the county defined specific 
proficiency expectations. Care now must be taken to ensure that attention to explicit indicators 
does not eclipse nurturing an in-depth understanding of mathematics. In contrast to Japanese 
standards, which place a heavy emphasis on “enabling children to deepen their understanding of’ 
various mathematical concepts, the MCPS indicators tend to be written at a much more specific 
level, making them appear to be more procedural — as opposed to conceptual — in nature. 
Although making the indicators clear, specific and measurable is very important, it also can 
create a “checklist” mentality where teachers are concerned more with coverage than they are 
with cultivating deep understanding of mathematics. To counteract this tendency, the county may 
wish to connect more tightly the process and content indicators. One approach would be to make 
the connections explicit in the clarifying examples. 

Following is a summary of a strand-by-strand comparison between the MCPS Framework and 
the benchmark standards: 

Elementary level: The MCPS Curriculum Frameworks tend to compare favorably with the 
benchmark documents at the elementary level. Reviewers noted that the MCPS indicators are 
particularly strong with respect to geometry, statistics and probability 

• Algebra , Patterns and Functions. By the end of grade 5, the expectations for MCPS, Japan 
and Arizona are roughly the same, with all requiring students to have a sense of variables, 
write and evaluate simple algebraic expressions in one variable, and write formulas for 
patterns using algebraic notation. 

• Geometry. MCPS indicators are stronger than the corresponding standards of Japan and 
Arizona through the end of grade 3 and tend to be roughly equivalent at grades 4 and 5. 
MCPS indicators provide more specificity and tend to be more developmental in their 
presentation than are the standards in either the Arizona or Japanese documents. This 
added detail provides helpful guidance for teachers and parents and places a solid, but not 
overwhelming, emphasis on geometric development in the early grades. 

• Measurement. Through grade 3, MCPS indicators are roughly equivalent to those from 
Japan and Arizona with respect to length, weight, time and volume, and a bit stronger with 
respect to estimation and error in measurement. They introduce the concepts of measuring 
temperature, perimeter and area a bit earlier. At the 4 th and 5 th grade levels, the MCPS 
indicators are comparable to the Arizona standards but not quite as rigorous as those from 
Japan. Japan, for example, introduces students to the concepts of “ratio units” — e.g., 
miles per hour — and measurement of area, including the formula for area of a circle. 
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• Statistics. This is a robust strand in the MCPS Framework, generally more rigorous than 
Japan’s. At grades 4 and 5, however, the Arizona standards are somewhat more 
demanding, requiring students to use box and whisker plots, scatter plots, and Venn 
diagrams. 

• Probability. MCPS introduces the basic ideas of probability and the definition of 
probability as a fraction between 0 and 1 earlier than Japan. MCPS is comparable to 
Arizona, but somewhat more explicit in that specific counting techniques, such as 
organized lists and tree diagrams, are mentioned in the MCPS Curriculum Framework. 

• Number Relationships and Computation. Both Japan and Arizona introduce addition and 
subtraction of common fractions earlier than does MCPS. Japan, at grades 4 and 5, 
requires its students to have completed multiplication and division of decimal fractions, 
and addition and subtraction with different denominators. 

Middle School Level: While the MCPS indicators still fare relatively well compared to the 
benchmarks at the middle school level, some nontrivial issues appear — particularly with respect 
to the complexity of the content introduced to all students by the end of grade 8. Reviewers’ 
concerns cluster in two areas: Algebra, Patterns and Functions; and Number Relationships and 
Computation. In conducting this analysis, reviewers introduced a third benchmark — Achieve’ s 
Foundations for Success expectations for the end of grade 8 — holding MCPS to a particularly 
high standard. 

• Algebra, Patterns and Functions. The benchmark documents appear to push further into 
algebra than either MCPS or Maryland by the end of grade 8. Japan’s standards and 
Achieve’ s Foundations for Success expect mathematically based solutions of simple 
simultaneous linear equations in two unknowns for all students by the end of 8 th grade, 
and all of the benchmarks expect some exploration of nonlinear functions and equations. 
Foundations for Success even suggests two approaches to the solution of quadratic 
equations — factoring and completing the square. In addition, Japan and Foundations for 
Success complete in grade 8 most of the essential work on linear functions and linear 
equations that MCPS reserves for Algebra I — with Foundations for Success also 
extending further to include nonlinear functions, operations on polynomials and the 
mathematical solution of quadratic equations. 

• Geometry. MCPS indicators are quite rigorous and exceed the expectations defined by 
Japan and Arizona with respect to coordinate geometry, transformations and especially 
the description of transformations via coordinate geometry. Three-dimensional geometry 
continues to be a strength. It is noteworthy that MCPS requires students in 8 th grade to be 
able to apply the Pythagorean Theorem, which is earlier than either Japan or Arizona, 
although it does not require proof of the Pythagorean Theorem, as do the Foundations for 
Success expectations. Foundations for Success also addresses special right triangles 
(30-60-90 and 45-45-90) by the end of grade 8. 
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• Measurement. MCPS indicators are roughly equivalent to the standards of Japan and 
Arizona, but less rigorous than those in Foundations for Success, where ratio units and 
conversion between such units are emphasized. 

• Statistics. MCPS indicators generally are stronger than those of Japan and Arizona, as 
indicated by their introduction of the line of best fit for points in a scatter plot. Both Japan 
and Foundations for Success, however, introduce contingency tables by the end of grade 
8, and Foundations for Success also introduces the concept of correlation. 

• Probability. MCPS introduces permutations and combinations earlier than Japan, Arizona 
or Foundations for Success, and MCPS also introduces the notions of dependent and 
independent events earlier than Arizona. 

• Number Relationships and Computation. MCPS indicators are on par with Arizona’s 
standards and more ambitious in the treatment of square roots and other roots than Japan. 
However, MCPS does not address explicitly irrational numbers in either its 8 th grade 
indicators or its Algebra I course. At the 8 th grade level, it is realistic to expect students to 

be familiar with irrational numbers like -/2 , V3 and K , meaning they understand that an 
irrational number cannot be represented as a quotient of integers and that it has a non- 
repeating decimal expansion. This omission is problematic since knowledge of irrational 
numbers is crucial for solving quadratic equations and for working with special right 
triangles. 

High School Level: For the sake of comparison with the benchmark documents, it is assumed the 
current expectation is that all MCPS students will take and successfully complete at least 
Algebra I and Geometry. With respect to the benchmark documents, comparisons are made 
based on the requirements defined for all students if they are to graduate from the school system. 
If an MCPS student completes Algebra I and Geometry, the student should have obtained a 
comparable level of mathematics proficiency to a student in Arizona. This MCPS graduate 
would be at a lower level of proficiency than the student graduating from a Japanese high school 
in certain aspects of number and trigonometry, but at a higher level of proficiency in data 
analysis. 

• Algebra, Patterns and Functions. MCPS indicators are roughly comparable to standards 
for all high school students in Japan and Arizona. MCPS tends to be somewhat more 
rigorous in at least one respect — its inclusion of applications and properties of 
exponentials. For example, MCPS expects all students completing Algebra I to be able to 
“make predictions based on an exponential curve of best fit.” MCPS should add indicators 
to the Algebra I course Framework that pertain to number relationships and computation. 
As students’ sophistication with mathematics increase in high school, it is appropriate for 
them to expand their knowledge and skills to include both rational and irrational numbers. 
One Algebra IB test item — which was identified as exceeding Maryland’s Core Learning 
Goals — addresses this content and is appropriate for students in an Algebra I course. The 
district’s Curriculum Framework for Algebra I needs to mirror this expectation. 
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• Geometry. MCPS has a strong set of geometry standards at the high school level. The 
district’s expectations that students be able to use coordinate geometry to understand 
transformations and to prove properties of geometric figures exceeds those found in the 
benchmark documents. The MCPS indicators also emphasize the construction of formal 
proofs by all students, and this expectation is either missing or not as explicit in Arizona’s 
and Japan’s standards. While MCPS and Arizona have roughly comparable expectations 
with respect to right triangle trigonometry, Japan’s standards are more rigorous — 
expecting all students to be introduced to the Law of Sines and the Law of Cosines. In 
MCPS, this is reserved for Honors Geometry students. 

• Measurement. The MCPS indicators are comparable to those of Japan and Arizona. 

• Statistics. MCPS places its expectations for statistics in its Algebra I course. MCPS 
indicators are closely aligned with Arizona’s standards — especially with respect to data 
analysis — and more rigorous than Japan’s. 

• Probability. As was true for statistics, MCPS expectations for probability are in its 
Algebra I course. There appears to be little progression in expectation from grade 8 to 
Algebra I MCPS indicators. For example permutations and combinations and independent 
versus dependent events are introduced in grade 8, but not expanded upon or extended in 
Algebra I. MCPS indicators are more demanding than Arizona’s and roughly equivalent to 
Japan’s. 

• Number Relationships and Computation. MCPS devotes little attention in Algebra I to 
concepts associated with real numbers and none to computations with real numbers. While 
Arizona devotes only slightly more attention to this strand, Japan devotes substantial 
attention to real numbers, including irrational numbers at grade 10. 

To improve clarity and increase the level of overall rigor in the mathematics Frameworks, 
consideration should be given to implementing the key recommendations, organized by standard, 
that are included in Achieve’ s technical report. That said, augmenting the content of the 
mathematics Frameworks to raise the level of rigor needs to be done judiciously to avoid 
sacrificing depth for breadth. If additions are not handled with great care, the result could be 
unwieldy, overstuffed Frameworks that could detract from the attention that must be given to 
developing conceptual understanding. This is a challenging task and will require MCPS 
educators to examine each strand for preK-12 to ensure progression is well developed and within 
each grade level to ensure the amount of content is manageable. (With respect to progression, 
MCPS can look to its own geometry strand where progression has been articulated in an 
exemplary fashion.) One approach to refining the Frameworks is to adapt Achieve’ s Framework 
for Success expectations for the end of grade 8 and back-map a trajectory of expectations from 
grade 7 on down to preK. 
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V MCPS should develop strategies for helping teachers and parents interpret the 
mathematical process standards at each grade level. 

As noted previously, MCPS has followed the state’s lead in its treatment of the mathematical 
process standards. The state standards for problem solving, communication, reasoning and 
connections are clear and comprehensive and worthy of adoption at the local level. While MCPS 
has been somewhat successful in infusing the substance of these four process standards into its 
indicators for the six content standards — via both language and the use of clarifying examples 
at the elementary and middle school grades — it is difficult to know what level of student growth 
is expected as students progress from grade to grade. For example, one would expect the level of 
sophistication of a student’s repertoire of problem-solving strategies to be different in high 
school than in the early grades of elementary school, but the expected growth is not spelled out 
in the Curriculum Frameworks. 

Of particular concern is the lack of a steady development of formal reasoning skills across the 
grades. MCPS sets high standards for its students, and mathematical reasoning is no exception. 
For example, Content Standard 9 (Reasoning) states that students are to be able to do such things 
as: follow and judge the validity of arguments by applying inductive and deductive thinking; use 
methods of proof, including direct, indirect, paragraph and/or contradiction; and use if-then 
statements to formulate valid arguments or proofs. These skills are explicitly captured in 
Geometry Indicator 2.2.3 but require development — and hence assessment — over many 
grades. For example, what does this indicator translate to in grade 4? In other words, what would 
one expect a student to be able to do in the 4 th grade if they were on track to eventually 
demonstrate this level of inductive and deductive reasoning in high school? 

The Arizona standards provide a nice model for the development of mathematical reasoning 
across the grades. By grade 3, for example, Arizona expects students to be able to understand 
and use quantifiers such as all, every, none, some and many. By the end of grade 5, they expect 
students to be able to construct simple valid arguments using if-then statements based on Venn 
diagrams. By the end of grade 8 students are constructing simple valid arguments using if-then 
statements based on syllogisms, and by grade 10, students are expected to determine the validity 
of arguments and formulate counterexamples. 

Another idea would be to include a compendium of sample problems that demonstrate the level 
of sophistication MCPS expects of its students at different grade levels with respect to problem 
solving, communicating, reasoning and making connections. These items likely would include 
problems that cut across multiple content domains while requiring students to solve a meaningful 
problem, communicate a problem solving strategy or apply a mathematical strategy to solve a 
problem outside the realm of pure mathematics. 
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•S The MCPS Frameworks could be focused more sharply to emphasize the most essential 
content. 

While alignment of the MCPS indicators to the benchmark standards at grades preK-8 generally 
is deemed strong, there is an essential difference between the MCPS Curriculum Framework and 
the Japanese standards in particular. The latter document tends to be focused more narrowly at 
each grade level, while the MCPS document tends to be more comprehensive. Because of this 
focus, the Japanese are able, for example, to emphasize the attainment of proficiency in 
computation at a slightly faster rate. Japanese students are expected to have completed 
multiplication and division of decimal fractions and adding/subtracting fractions with different 
denominators by the end of grade 5, while MCPS students are still tackling these skills in grade 
6. Since MCPS students are, during this time, gaining greater exposure to a breadth of topics 
including concepts related to probability and statistics, giving more attention to computation 
might entail some trade-offs. In addition, while there are similarities between the Japanese 
expectations and those of MCPS with respect to functions and algebra, the Japanese algebra 
standards become much more focused and ambitious in the middle school grades — requiring 
students to tackle the solution of linear equations and two-by-two linear systems by the end of 
grade 8. 

If MCPS students are to meet algebraic expectations comparable to those contained in Achieve’s 
benchmark standards, it would be helpful to shift some of the content now in Algebra I to grade 
8. This shift, in turn, would necessitate refining the scope and sequence of content in the middle 
school indicators. 

•S The county’s expectation with respect to the role of technology in the classroom is not 
clear. 

Although MCPS makes some general statements in its standards and its accompanying 
introductory material with respect to technology, it is not clear what the district’s philosophy is 
with respect to the appropriate role of technology in the classroom. Specifically, it is not clear 
what type of technology is deemed appropriate at different grade levels or the extent to which 
students are expected to perform routine, algorithmic tasks without the aid of a calculator. In the 
preK-8 standards, a clarifying example for Indicator 2.2.4 refers to the student using “a fa mi liar 
software program” to create symmetric figures. Does this mean that any such software meets 
district standards? There is no clear statement of the type of technology that should be available 
at different grade levels or the extent to which routine tasks — computation, graphing and 
eventually algebraic manipulation — might or might not be aided by a calculator or other means 
of computing technology. Reviewers urged more clarity in the wording of the indicators 
themselves and in clarifying examples, which could be quite explicit with references to 
technology and software products. 
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Major Findings: Semester Tests in Algebra I 
and Geometry 



Achieve, Inc. carried out a detailed study of the rigor and alignment of the county wide end-of- 
course examinations in Algebra I and Geometry to the Maryland High School Core Learning 
Goals. Achieve reviewed four mathematics tests: 

• Algebra IA (first semester of Algebra I), Form 1 (2001); 

• Algebra IB (second semester of Algebra I), Form 1 (2002); 

• Geometry A (first semester of Geometry), Form 1 (2001); and 

• Geometry B (second semester of Geometry), Form 1 (2002). 

The semester tests are designed to assess the MCPS High School Curriculum Framework for 
Algebra I and Geometry. Because Maryland’s High School Core Learning Goals serve as the 
foundation for both the indicators articulated in the MCPS Framework and for the state’s High 
School Assessments, Achieve conducted its alignment analysis against the state’s Core Learning 
Goals. 

Each of the four semester tests was analyzed using Achieve’ s assessment-to-standards protocol. 
For the purpose of this study, test items were compared with indicators from the three Maryland 
Core Learning Goals for mathematics: 

• Functions and Algebra; 

• Geometry Measurement and Reasoning; and 

• Data Analysis and Probability. 

The Assessment Limits and Skill Statements that are included within the Core Learning Goals 
were used by Achieve’ s reviewers to clarify the intent of the indicators and define the testing 
boundaries inherent in each indicator. 

All of the assessments reviewed consist of both selected-response (multiple-choice) items and 
constructed-response items. While each test has a different number of items — varying from 34 
items on the Algebra IA test to 49 items on the Geometry A test — and a different mix of 
selected-response and constructed-response items, the time allotted for each assessment is two 
hours. The point value for selected-response items is consistent across tests: two points per item. 
The point values of constructed-response items vary from two to eight points per item, with 
Maryland’s High School Assessment rubric being used to score a portion of the constructed- 
response items. 

Each test is intended to assess discrete units of instruction: Algebra IA tests Units 1-5; Algebra 
IB tests Units 5-10; Geometry A covers Units 1-3; and Geometry B covers Units 4-7. As is 
consistent with strong instructional practice, students are called upon to use mathematical tools 
and/or technology during the course of the MCPS semester examinations. Graphing calculators 
are required on the Algebra IA and Algebra IB exams, while calculator use is permitted, but not 
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required, on the Geometry A and Geometry B tests. Students also need a compass and 
straightedge for both portions of the geometry assessment. In addition, students are provided 
with an extensive Formula Reference Sheet, including a supplementary Geometry Formula 
Sheet. 

Strengths 

• The match between content and performance as measured by the MCPS semester tests for 
Algebra and Geometry and the Maryland Core Learning Goals generally is strong. 

Over 93 percent of the first semester Algebra IA items assess content in the Maryland Core 
Learning Goals, while 84 percent of the items assess the performances described in the goals. 
Reviewers found the content match between Algebra IB and the Maryland goals to be less 
consistent than the content match between Algebra IA and the state goals. This is actually a 
positive finding: The content match for Algebra IB, administered at the end of the second 
semester, is less strong (70 percent) because the test assesses content that extends beyond what 
the state includes in its goals. For example, two items on the Algebra IB test require students to 
solve quadratic equations — an expectation of the district’s Framework but not of the state goals. 
Similarly, the state limits its expectation with respect to students being able to simplify algebraic 
expressions to polynomials in one variable, but the district does not place this limitation on its 
indicators. The performance match between the items on the Algebra IB test and the state goals 
was stronger than the match of content with 94 percent of the items being consistent. 

The Geometry A and B tests also align well with the content and performance expectations 
defined in the Maryland Core Learning Goals. Almost 97 percent of the items were judged to 
have clear and consistent alignment with the content of the Core Learning Goals. Only two of the 
88 items across these two tests were inconsistent with the content of the Core Learning Goals. 
The match between the performances demanded by the items and the performances called for by 
the Core Learning Goals also is strong, although not quite as robust as the match for content. 
Approximately 92 percent of the items were fully or partially aligned to the goals, with most of 
these (88 percent) being a full match. Only four of the 88 items were found not to match the 
performances expected by the Core Learning Goals. 

• MCPS semester tests in Algebra and Geometry contain a solid mix of multiple-choice and 
constructed-response items. 

MCPS deserves credit for including a significant number of constructed-response items on their 
tests. The inclusion of brief and extended constructed-response items, along with multiple-choice 
items, enhances the challenge of the assessments. MCPS is sending an important signal to its 
students, teachers and community that it places value on students’ ability to reason, problem 
solve and communicate mathematically. Brief constructed-response items have value because 
they can tap a different level of skill than multiple-choice items, yet they also can assess some 
essential content effectively and more efficiently than extended constructed-response items. The 
power of extended constructed-response items lies in their giving students the opportunity to 
demonstrate a deeper, more complex understanding of content than can other item formats. For 
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example, they allow the possibility of assessing higher- level skills such as students’ abilities to 
construct graphs, formulate equations, construct proofs and explain their reasoning. They also 
provide a venue to assess student proficiency in integrating multiple aspects of the discipline. 

Slightly less emphasis tends to be given to constructed-response items on the Algebra IB test, 
when compared with Algebra IA. A majority of points (52 out of 100) on the Algebra IA test 
comes from constructed-response items, while only 45 out of 95 points (47 percent of the total 
points) on the Algebra IB test are attributable to constructed-response items. 

Reviewers found a somewhat greater emphasis on constructed-response items in the Geometry B 
test as compared with the Geometry A test. A majority of points (70 out of 120, or 58 percent of 
the points) on the Geometry B test come from constructed-response items, while about 48 
percent of the points (or 61 out of 127 points) on the Geometry A test are attributable to 
constructed-response items. 

• The district’s semester tests in Algebra and Geometry are well crafted. 

No technical flaws were found on either of the Algebra I tests and only a few were detected on 
the Geometry tests. Specifically, three items from the first semester test (Geometry A) and one 
item from the second semester test (Geometry B) — in total, only about 4 percent of all items on 
both tests — were flagged as potentially flawed. (Reviewers would have been able to make 
better judgments on possible defects in constructed-response items had they been provided with 
examples of student responses for various scores.) 

Examples of common technical flaws Achieve has encountered in mathematics tests from around 
the nation include having no or multiple correct answers, misleading graphics, incorrect labels on 
figures, and items where the reading is more a factor than the mathematics. Defective items may 
cause the district to end up with an incorrect perception of precisely which concepts or skills are 
causing students difficulty. 

All in all, reviewers found MCPS to have few problematic mathematics items; the county has 
been remarkably successful in this aspect of test development. In contrast, Achieve has reviewed 
state tests where up to 25 percent of items have been found to be technically flawed. 

• District tests measure the full span of the Maryland High School Core Learning Goals for 
Mathematics. 

When examined collectively, Algebra IA and IB and Geometry A and B do a good job of 
assessing, in a balanced way, the content in Maryland’s High School Core Learning Goals. In 
addition, coverage of the indicators generally is good with respect to all three of the Maryland 
goals. The Algebra tests assess all of the Goal 1 (Functions and Algebra) indicators and four of 
the six Goal 3 (Data Analysis and Probability) indicators. The Geometry tests assess all nine of 
the Goal 3 (Geometry, Measurements and Reasoning) indicators. The content focus of items on 
the Geometry A and B tests is different, which is as it should be given that the intent of the 
assessments is to assess students’ mastery of material defined in different units. (This also is true 
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for Algebra IA and IB.) For example, all of the items assessing coordinate geometry, congruent 
and similar figures, right triangle trigonometry, and special right triangles occur in the second 
semester test. However, taken collectively, the two tests provide a balanced assessment of 
Maryland’s Core Learning Goal 2 (Geometry, Measurements and Reasoning). 

Areas for Improvement 

S If MCPS wants to ensure that its students are prepared to be successful in advanced 
courses including calculus, the district will need to consider increasing the rigor of its 
semester tests. 

The vast majority of items on the Algebra IA and IB tests were at the two lowest levels of 
demand — Levels 1 and 2. Reviewers were disappointed that more Level 3 items were not 
included. While the Algebra IB test is more challenging than the Algebra IA test in that it 
includes a number of items assessing content beyond Maryland’s High School Core Learning 
Goals, the level of demand of items on the Algebra IA test tends to be somewhat higher. While 
the majority of items on the Algebra IA test are rated Level 2, with some Level 3 items also 
identified, most of the items on the Algebra IB test are Level 1, and no Level 3 items are 
included. In addition, the constructed-response items on the first semester Algebra IA test tend to 
be more challenging than those on the second semester test. On the Algebra IA test, eight of the 
10 constructed-response items were rated as Level 2, with the two remaining items being Level 
3. In comparison, the constructed-response items on the Algebra IB test were evenly split 
between Levels 1 and 2. The result is that the Algebra IB test, while more challenging in terms of 
its content, actually asks less of students in terms of higher order thinking. Infusing items that 
demand complex thinking is an issue MCPS will want to address, regardless of students’ 
terminal high school course in mathematics. 

Similarly, nearly all of the items on the Geometry A and B tests are at the two lowest levels of 
demand — Levels 1 and 2. Only one item — an unscaffolded proof on the Geometry A test — 
received a Level 3 rating. An increased number of Level 3 items would raise the rigor of the 
tests. The Geometry B test includes a higher percentage of Level 2 items when compared with 
the Geometry A test (68 percent as compared with 54 percent) and a correspondingly lower 
proportion of Level 1 items (32 percent as compared with 44 percent). (Geometry B, however, 
includes two constructed-response items that were rated as Level 1, while Geometry A includes 
all Level 2 constructed-response items and one Level 3 item — an unscaffolded proof.) 

Although the Geometry B test contains more Level 2 than Level 1 items, the items still tend to be 
heavily skill-based and procedural. The test contains very few problems presented to students in 
a “real-world” context. Rather, the items — while addressing important content — require 
students to apply their geometric understandings in more of a pure mathematical problem- 
solving context. In addition, students are seldom required to determine what geometric skills and 
concepts they need to employ to solve a particular problem. 

Ratcheting up the level of rigor of items in mathematics can be accomplished in a number of 
ways. One way is to reduce the scaffolding in some items. Rather than breaking down the 
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problem into a series of steps and leading students to a solution one step at a time, items can 
place more of a burden on students to come up with the “whole solution” and generate the steps 
themselves. For example, as also was the case with the Geometry A test, students are provided 
with a variety of forms of support or scaffolding on the Geometry B test, making it less 
challenging than it would be if less support had been present. To be specific, the only proof on 
the Geometry B assessment is a short, straightforward similarity proof that is made even easier 
by providing students with two of the four statements and two of the four reasons. Such 
scaffolding should not be necessary by the end of a second semester Geometry course — 
particularly given that students are expected on the Geometry A test to do one unscaffolded 
proof, in addition to two scaffolded proofs. 

Another way to increase the level of intellectual demand is to have students rely less on formulas 
provided to them and instead figure out which formula or combination of formulas best fits a 
given situation. The state supplies a comprehensive and detailed Formula Reference Sheet, 
which MCPS augments with one of its own. With access to such aids, students need to 
internalize very few formulas. For example, they need no conceptual understanding of the 
trigonometric functions or of special right triangle relationships if they know such formulas will 
be provided to them. Granted, the state makes these formulas available to students on its 
statewide assessments, but reviewers recommend that MCPS give some consideration as to what 
it expects students to know when they have successfully completed a high school Geometry 
course. Should these students know a core number of basic geometry formulas and understand 
why these formulas work? Or should they just be able to select the correct formula and substitute 
values into it? If the answer is the former, then MCPS should consider the level of support it 
provides to students on its semester assessments. In the end, a combination of both probably 
makes sense, with some of the more basic formulas memorized and others supplied. 

Rigor also can be injected by including more contextual problems, which require students to 
apply concepts and skills in somewhat unfamiliar, less scripted ways. Following is an illustration 
of how an item could be made more challenging by providing it with a real-world context. One 
MCPS item, as currently written, requires students to use formulas, supplied to them in the 
Formula Reference Sheet, and substitute values into the formulas to calculate the measure of an 
angle. If the item were instead constructed to require students to determine the angle in a 
practical situation, students would be required to select an appropriate formula(s) to use, 
calculate the value of the function and then determine the angle measure. The use of such 
strategies to contextualize a subset of test items — both selected-response and constructed- 
response — would add to the level of challenge of the tests. 

S The district should clarify scoring procedures for constructed- response items. 

MCPS has prepared a helpful document, “Comments to Teachers Regarding Semester Exams,” 
that provides sample student responses and guidelines for awarding points for the constructed- 
response items. Ideally, item-specific rubrics — in addition to the High School Assessment 
Rubrics for extended and brief constructed-response items — should be available to teachers 
who will be scoring the constructed-response items on these assessments. It also would be 
helpful to teachers to have sample student responses at each of the scores for those constructed- 
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response items intended to be scored using the High School Assessment Rubrics and, to the 
degree feasible, for item-specific rubrics. It is not clear how scoring decisions are to be made for 
these items, and teachers countywide need to be able to apply consistent criteria in determining 
student scores. Fairness to students demands that scoring guides be carefully constructed and in 
the hands of all those who score the tests. 

If the district wants to ensure that all students will be prepared to take calculus at grade 12, it 
should continue to implement an instructional infrastructure — diagnostic testing and classroom 
data reports, support for students, and aligned professional development. 

If MCPS is to enable all of its students to succeed in calculus in grade 12, it will need to expand 
and sustain its systematic efforts to enact a standards-based instructional system from preK on. 
High standards require high support and in no subject area is that more the case than 
mathematics. As the district incrementally raises its standards, it will want to ensure that teachers 
receive the ongoing professional development they require to teach to higher standards, that 
teachers and school leaders have access to diagnostic tests and classroom performance data, and 
that students have access to a variety of quality supports that range from technology-driven 
tutorials to summer-school courses. 

Fortunately, high-quality resources are more available than ever before. Technology now makes 
it possible to administer diagnostic classroom tests and retrieve classroom data in a timely 
fashion. Targeted small group instruction, after-school and/or Saturday classes, and summer 
school can all help keep students on track. And evidenced-based results have identified effective 
models for enhancing teachers’ mathematical knowledge and pedagogical skills. 

To support these efforts, MCPS may wish to consider joining Achieve ’s Mathematics 
Achievement Partnership. Its coordinated approach of providing world-class standards for the 
end of grade 8 and back-mapping these down through the elementary grades, high-quality 
teacher support and curriculum guidance, and ongoing diagnostic tests along with an end-of- 
course grade 8 assessment, could help MCPS put in place the infrastructure it needs for 
strengthening and updating its mathematics program for preK-12. 
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Appendix 



Expert and Staff Biographies 

Achieve relied on the expertise of nationally respected experts in academic content, standards, 
curriculum and assessment design to inform and conduct the standards benchmarking and 
alignment of assessments to standards. 

Achieve Senior Project Director 

Jean Slattery 

Jean Slattery has been a consultant for Achieve since 1999 and currently serves as director for 
the Benchmarking Initiative. She was supervising director of curriculum development and 
support in Rochester, N.Y., from 1989 to 1997, with responsibility for overseeing the work of all 
subject-area directors in the K-12 instructional program. Her earlier responsibilities as a district- 
level administrator included serving as director of the middle school (1987-89) and junior high 
(1985-87) programs. During this period, she initiated Teachers as Partners, a peer-coaching staff 
development program funded by the Ford and Matsushita (Panasonic) Foundations. 

Slattery also is a peer consultant on standards and assessment for the U.S. Department of 
Education. She has served as a consultant to the Washington, D.C., school district; San Diego 
Unified School District; a Washington state consortium of rural schools; and the Alabama and 
Illinois Departments of Education. Slattery also has worked for the Council for Basic Education 
on projects involving the Flint Community School District, the Nevada Education Department 
and the Cleveland Municipal School District. 

Slattery received a bachelor’s degree in chemistry from Albertus Magnus College, a master’s 
degree in science education from Yale University and a doctorate in science curriculum from the 
University of Rochester. 

English/Language Arts 

JoAnne Thibault Eresh 

JoAnne Thibault Eresh is a senior associate at Achieve, where she leads the English/language 
arts aspects of the Standards-to-Standards Benchmarking and Assessment-to-Standards 
alignment reviews. She taught writing at the university level and English at private and public 
high schools in St. Louis, Mo., and in Fitchburg, Mass. She began her work in curriculum design 
and performance assessment in 1979 under Superintendent Richard C. Wallace Jr. and from 
1981 to 1994 was director of the Division of Writing and Speaking for the Pittsburgh Public 
Schools. During that time, she directed The Pittsburgh Discussion Model Project, funded by the 
Rockefeller Foundation and part of the CHART network, and later directed the imaginative 
writing part of the ARTS Propel Project, a joint project with Harvard’s Project Zero and the 
Educational Testing Service. She was the Pittsburgh district coordinator for the New Standards 
Project and wrote the teachers’ guides for the New Standards English Language Arts portfolios. 
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In 1995, she was one of the original resident fellows at the Institute for Learning at the 
University of Pittsburgh’s Learning Research and Development Center, and coordinated the New 
Standards Linking Projects. From 1997 to March 2001, she was the coordinator of staff 
development in Community District Two in New York City, where she was responsible for the 
hiring, training and coordinating that district’s staff development group. 

Standards Reviewers 

Elizabeth Haydel 

Elizabeth Haydel was the project manager for Indiana University’s Center for Innovation in 
Assessment in Bloomington and the project coordinator for the Center for Reading and 
Language Studies. A graduate of Stanford University with a degree in American Studies, Haydel 
also holds a master’s degree in language education from Indiana University. She is currently an 
English/language arts consultant for the Ohio Department of Education. 

Haydel has taught reading and writing to high school students who had failed Indiana’s statewide 
achievement test and “Reading in the Content Areas” for Indiana University’s Language 
Education Department. She has co-authored a number of reading workbooks for children, 
including Steck-Vaughn Think- Alongs: Comprehending While You Read program. Haydel has 
written test passages and items for various state assessment and test preparation programs. 

Carol Jago 

Carol Jago teaches English at Santa Monica High School in Santa Monica, Calif, and directs the 
California Reading and Literature Project at UCLA. She also edits the California Association of 
Teachers of English quarterly journal, California English. Jago has written a weekly education 
column for the Los Angeles Times, and her essays have appeared in English Journal, Language 
Arts, NEA Today, The Christian Science Monitor and other newspapers across the nation. She 
has served as director of the National Council of Teachers of English (NCTE) Commission on 
Literature and currently is a member of NCTE’ s Secondary Section. NCTE has published her 
books Nikki Giovanni in the Classroom, Alice Walker in the Classroom and Sandra Cisneros in 
the Classroom. Her other books for teachers. With Rigor for All: Teaching the Classics to 
Contemporary Students, and Beyond Standards: Excellence in the High School English 
Classroom and Cohesive Writing: Why Concept Is Not Enough are published by Heinemann. 

Susan Pimentel 

Susan Pimentel, co-founder of StandardsWork, a nonprofit education consultancy, specializes in 
standards-driven school reform and works as an education writer, analyst and coach. After 
earning a bachelor of science degree in early childhood education and a law degree from Cornell 
University, Pimentel served as senior policy adviser to Maryland Governor William Donald 
Schaefer and subsequently as special counsel to former Superintendent John Murphy in Prince 
George’s County, Md., the nation’s 16 th -largest school district. 

In recent years, her work has focused on helping communities and schools throughout the 
country work together to advance meaningful and enduring standards-based education reform. 
This includes the development and implementation of rigorous grade-by-grade standards; results- 
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based evaluation systems; diagnostic assessments; and a powerful new reporting tool, called The 
Results Card, which helps communities and educators stay focused on student achievement. 
StandardsWork focuses on building the system from the inside out, equipping school leaders 
with the resources and support they need to sustain the process of continuous improvement, close 
community collaboration and data-driven results. 

Beyond her work with districts, Pimentel has emerged as an expert in standard setting. The 
Fordham Foundation has ranked the states in which Pimentel has coordinated the standards- 
setting effort as having among the best content standards in the country. California and Arizona 
were ranked first and second, respectively. She also has helped Maryland revise its 
English/language arts and social studies standards, raising that state’s overall rank from 43 rd in 
1998 to 10 lh in 2000. Pimentel is co-author with Denis P. Doyle of the best-selling book and CD- 
ROM, Raising the Standard: An Eight Step Action Guide for Schools and Communities. 

Assessment Reviewers 

Sue Craig 

Sue Craig is a retired educator. She has been an outreach educator for the Oregon Museum of 
Science and Industry and a classroom management trainer for both the California Teachers 
Association and the Oregon Education Association. Craig is a member of the California State 
Board of Education State Assessment Language Arts Content Review Panel, a member of the 
California State Board Performance Level Setting Panel, a member of the California State Board 
panel to develop writing prompts and scoring guidelines, and a member of the California 
Teacher Credentialing panel on reciprocity. 

A classroom teacher for 30 years, Craig’s other professional activities include teaching special 
education, U.S. history, social science, English and English as a second language. 

Marshall George 

Marshall George is a faculty member in the Graduate School of Education at Fordham 
University in New York City. At Fordham, he serves as director of teacher education and is an 
assistant professor of English and literacy education. Before moving to higher education, 
Marshall taught high school English in North Carolina for five years and was a middle and high 
school English teacher at an international school in Brazil for three years. He has a doctorate in 
English education from the University of Tennessee, a master’s degree in secondary curriculum 
and instruction from the University of Southern Mississippi, and a bachelor’s degree in English 
education from The University of North Carolina at Chapel Hill. George has published 
extensively in the field of adolescent literature and English education and serves as a language 
arts and literacy consultant in numerous school districts around the New York City metropolitan 
area. 
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LeRoy Mi ller 

LeRoy Miller is in his 28 th year of teaching 1 1 th grade English at Sidney High School in Ohio, 
where he also serves as English department chair for a nine-member department. Miller also is a 
teacher of Advanced Placement and honors American literature and an adviser for the Academic 
Competition Team. In addition, he served as a commissioner on the Governor’s Commission on 
Student Success in 2000 and is a member of Ohio’s Academic Content Standards Writing Team 
for English/language arts. Miller received a bachelor’s degree in English from Fairleigh 
Dickinson University and a master’s degree in secondary English from Miami University. 

Recardo Sockwell 

Recardo Sockwell is the coordinator for the Office of Program Evaluation of Fairfax County, 
Virginia Public Schools. He also is a consultant for the U.S. Department of Education’s Office of 
Elementary and Secondary Education. Before joining the Fairfax County Public School System, 
he served as the executive director for research, Standards and accountability with the Memphis, 
Tenn. School System, as a research analyst and in-service coordinator with the Charlotte - 
Mecklenburg, North Carolina Schools, as an interim curriculum coordinator with the Chapel- 
Hill/Carrboro City, North Carolina School System, as a high school English teacher for the 
Winston-Salem/Forsyth County, North Carolina School System and as an English instructor for 
the Forsyth Technical Institute in Winston-Salem, N.C. 

Sockwell received a bachelor’s degree in English from Johnson C. Smith University in Charlotte, 
S.C., and a master’s degree in English and a doctorate in curriculum and instruction from the 
University of North Carolina at Chapel Hill. 

Mathematics 

Kaye R. Forgione 

Kaye R. Forgione began consulting work with Achieve in 2000 and joined Achieve as senior 
associate for mathematics in March 2001. Forgione’s primary responsibilities are managing and 
providing intellectual leadership to Achieve’ s standards and benchmarking work involving math. 
Before joining Achieve, she served as assistant director of the Systemic Research Collaborative 
for Mathematics, Science and Technology Education project at the University of Texas at 
Austin. Her responsibilities also include administrative and design responsibilities for UTeach, a 
collaborative project of the College of Education and the College of Natural Sciences to train and 
support the next generation of math and science teachers in Texas. Prior to her work at the 
University of Texas, Forgione was director of academic standards programs at the Council for 
Basic Education, a nonprofit education organization based in Washington, D.C. 

Prior to joining the Council for Basic Education in 1997, Forgione worked in the K-12 arena in a 
variety of contexts including district-level curriculum supervisor for math, assessment and 
gifted/talented programs. She also was team leader for assessments and task development and 
supervisor in the areas of assessment, school profiles and educational indicators at the Delaware 
Department of Education; senior research associate for development at the Delaware Education 
Research and Development Center at the University of Delaware; and a high school math 
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teacher. Her personal portfolio of work includes math-related and policy development work in a 
number of states (including Nevada, Maryland and Hawaii) and school districts (including 
Cleveland; East Allen County, Ind.; and Los Angeles) and in partnership with a number of 
organizations (including Achieve, Inc., George Washington University and the Institute for 
Educational Leadership). Forgione earned a doctorate from the University of Delaware. 

Standards Reviewers 

Susan K. Eddins 

Susan K. Eddins has taught students in kindergarten through college for more than 30 years — 
24 years at the high school level. She is the recipient of several honors for her teaching, 
including the Presidential Award for Excellence in Mathematics Teaching, and she is a National 
Board Certified Teacher in Adolescent and Young Adult Mathematics. Eddins is a faculty 
member, an instructional facilitator, and the curriculum and assessment leader in math at the 
Illinois Mathematics and Science Academy, where she has taught since the school’s inception in 
1986. 

She has served in leadership capacities in several professional organizations and currently is a 
member of the board of directors of the National Council of Teachers of Mathematics (NCTM). 
Eddins holds bachelor’s and master’s degrees in mathematics. 

Eddins was a member of the 9-12 writing group for NCTM’s Principles and Standards for 
School Mathematics. She is co-author of a chapter in NCTM’s Windows of Opportunity and is a 
co-author of UCSMP Algebra. She is a past panel member and editor of NCTM’s Student Math 
Notes and has authored several articles in refereed journals. Over the past four years, in addition 
to numerous workshops and presentations, her most extensive work has been in the area of 
standards development, standards review and alignment of standards to assessments. For 
Achieve, she has reviewed academic standards or assessments from Alaska, Illinois, Indiana, 
Minnesota, New Jersey, Oregon, Pennsylvania, Texas and Washington. 

Donald R. King 

Donald R. King is an associate professor of mathematics at Northeastern University. Previously, 
he was a visiting assistant professor at Salem State College, a visiting assistant professor at the 
University of California at San Diego and a high school math teacher in Boston. King is a 
member of the Mathematical Association of America, American Mathematical Society and 
National Association of Mathematicians. King is active in professional and community service: 
He was a parent member of the Mathematics Focus Group for Boston Public Schools in 1997; 
director from 1993 to 1994 of NUMATH, Northeastern University’s program to foster minority 
mathematical achievement and talent in high school; an adviser to algebra- in-middle- schools 
projects from 1990 to 1992; a review panelist for three years for Ford Foundation postdoctoral 
fellowships for minorities; and an adviser to Massachusetts’ pre-engineering program for 
minorities from 1988 to 1991. King recently gave a speech at the American Mathematical 
Society’s Special Session on Teaching Mathematics in the New Millennium titled “Changing 
school outcomes: raising standards and promoting equity,” and he has advised Achieve on the 
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quality of standards and assessments in a number of states including Minnesota, New Jersey, 
Oklahoma and Texas. 

King received his doctorate in mathematics from the Massachusetts Institute of Technology. 

R. James Milgram 

R. James Milgram is a mathematics professor at Stanford University in California. He also has 
been a visiting professor at the mathematics institute in Barcelona in June 1998, at Northwestern 
University in March 1997 and at the University of Montreal in May 1995. 

Recent lectures include those given at Santa Cruz University, University of Chicago, University 
of Illinois and Indiana University. Milgram, together with Gunnar Carlsson, Ralph Cohen and 
Steve Kerckhoff, revised the California Mathematics Standards and Framework under the 
direction of and for the California State Board of Education. He also was a member of the 
content review panel for math curricula for the California Textbook Adoption in 2000 and has 
been a member of the National Science Foundation Panel on Mathematics and Robotics since 
May 2000. 

Milgram is a member of the Mathematics Achievement Partnership, a project of Achieve, Inc., to 
create a common set of expectations and assessments for math in the middle grades, and was an 
expert math reviewer for Achieve’ s study of Texas’ proposed objectives and the Texas Essential 
Knowledge and Skills student expectations for the second Texas Assessment of Academic Skills 
study. 

He received a bachelor’s degree in science and a master’s degree in science from the University 
of Chicago and a doctorate from the University of Minnesota. 

Lynn A. Steen 

Lynn A. Steen has been a member of the St. Olaf College faculty since 1965 and currently is the 
director of institutional research and planning. He is leader of the quantitative literacy initiative 
of the National Council on Education and the Disciplines at the Woodrow Wilson Foundation 
and a member of the Mathematics Achievement Partnership, a project of Achieve, Inc., to create 
a common set of expectations and assessments for math in the middle grades. Steen is former 
president of the Mathematical Association of America and former chair of the Council of 
Scientific Society Presidents. He has reviewed math standards for Achieve from Massachusetts, 
Oklahoma and Texas. He received his doctorate in mathematics from the Massachusetts Institute 
of Technology. 

Assessment Reviewers 

Susan K. Eddins 

Donald R. King 
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Mary Lynn Raith 

Mary Lynn Raith currently is a mathematics specialist in the Division of Instructional Support of 
the Pittsburgh Public Schools. As such, her responsibilities include leadership roles in curriculum 
development; textbook selection; design of alternative assessments — both performance tasks 
and portfolio development; in-service program design and implementation; in-class support of 
teachers; and coordination of math programs across levels and schools. She has special 
responsibility for middle schools: Raith is the co-director of the Pittsburgh Reform in 
Mathematics Education project, a K-12 professional development system that includes both in- 
class support from demonstration teachers and a comprehensive series of in-service sessions 
focused on increasing the depth and breadth of teachers’ math knowledge. 

Prior to this position, Raith was a math supervisor (1986-96) in Pittsburgh and a middle school 
math specialist in grades 6 through 8 (1970-86), working with remedial as well as gifted 
students. She has designed and presented — at local, regional and national conferences — 
sessions on the infusion of algebraic thinking, geometric reasoning, statistics and probability, and 
problem solving in the K-8 math program. In summer 1987, Raith was chosen to attend a 
Michigan State University (MSU) honors teachers workshop and since then has been involved 
with the implementation, piloting and in-servicing of MSU programs, including the Middle 
Grades Mathematics Program in Pittsburgh and the Connected Mathematics Project in Pittsburgh 
and other school districts across the nation. 

She has been involved with a number of national projects, including the Assessment 
Communities of Teachers project, which supported middle school teachers in the use and 
development of assessment tools in their classrooms, and the Alternative Assessment in 
Mathematics project, which defined criteria that identified adequate progress in math 
achievement for Title I students. 

Raith was a middle school leader for the New Standards Portfolio project, which designed a 
portfolio system for use in both classroom and district assessment. She also was a middle school 
leader for the New Standards Reference Examination development project, which included 
heading up the task and rubric-development, scoring, and standard- setting teams. In partnership 
with the New Standards math team she has presented at math leadership conferences in New 
York; Washington, D.C.; and Bucks County, Pa. Raith joined the National Council of Teachers 
of Mathematics Academy faculty and has presented the Principles Academy at NCTM- 
sponsored national conferences. She also has worked extensively with the National Center on 
Education and the Economy (NCEE) and the America’s Choice school design on designing and 
delivering professional development on standards-based math curriculum, instruction and 
assessment and has presented at the annual NCEE national conference. Raith received a 
bachelor’s degree in mathematics from Indiana University at Pittsburgh and a master’s degree in 
mathematics education from the University of Pittsburgh. 
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